4

I'm new to apache spark.

Is it possible to configure multi cluster spark without hadoop?

If so, can you please provide the steps. I would like to create clusters on Google Compute Engine (1-master, 1-worker)

user4290511
  • 101
  • 6

1 Answers1

1

As explained in the comments by @Emre, there is a python script on github which you can use to set up a cluster of whatever size you like: Spark Cluster on Google Compute Engine | Ido Green. It allows you to specify the instance type to use for the workers (slaves) and separately what to use for the master.

Note that it uses gcutil, which is deprecated in favor of gcloud compute

nealmcb
  • 408
  • 5
  • 7