This setup is for submitting Apache Spark jobs to the Amazon EMR cluster from a remote machine, such as the ec2 instance from the Elastic beanstalk instance. This might be useful for the services running in ebs which would also interact with the spark engine for computation.

Short Description

To submit Spark jobs to an EMR cluster from a remote machine, the following must be true:

  1. All Spark and Hadoop binaries are installed on the remote machine.
  2. The configuration files on the remote machine point to the EMR cluster.

Resolution
Confirm…

Harshit Saklecha

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store