O que é o Spark Cluster:

Pré-requisitos:

  • Ubuntu 18.04 (3 máquinas virtualizadas no VirtualBox ou outra plataforma);

Passo 1 (Maquina incial):

Passo 2 (Clonagem e iniciação das maquinas):

sudo nano /etc/hostname
ip addr
sudo nano /etc/hosts
sudo reboot
$ sudo apt-get install software_properties_common
$ sudo add-apt-repository ppa:webupd8team/java
$ sudo apt-get update
$ sudo apt-get install openjdk-11-jdk
ssh-copy-id user@slave1
ssh-copy-id user@slave2

Passo 3 (Instalação do Spark):

export SPARK_MASTER_HOST='<MASTER-IP>'export JAVA_HOME=<Path_of_JAVA_installation>

--

--

--

Love podcasts or audiobooks? Learn on the go with our new app.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
João Saraiva

João Saraiva

More from Medium

Tea.V1 Design Plan

Biased Algorithms

Part 2: Goldie in the Goldilocks

Open Call for Papers for 2022 ICLR Workshop on Collective Learning