Dev Environment Setup

Requirements for developing it via docker

Docker is a shortcut for quickly launching Solr and admin dashboard using prebuilt image. If you wish to install Solr natively, then skip docker and install solr from http://archive.apache.org/dist/lucene/solr/7.1.0/

Launch Solr and Banana Dashboard

Using Docker:

docker run  -p 8983:8983 --user sparkler -it uscdatascience/sparkler

Using Solr natively:

Follow instructions highlighted by this URL

In the default setting, sparkler tries to connect with solr at http://localhost:8083/solr and the dashboard at http://localhost:8083/banana

Building the Project

Obtaining the Source code for the first time

git clone git@github.com:USCDataScience/sparkler.git
cd sparkler

Launch a terminal and cd to the root of project.

Building whole project:

git pull origin master
mvn clean package

The whole project includes API, App, and Plugins. It also runs all the test cases.

To build the core project excluding plugins:

mvn clean package -Pcore

When build task is SUCCESS, it creates a build directory with the following structure

build/
   ┝---bin/               -- Useful scripts
   |   └- sparkler.sh     -- Command line interface
   ┝-- conf/              -- All the config files
   ┝-- plugins/           -- All the plugin jars
   └--- sparkler-app*.jar -- Application code except plugins

Running a test crawl

build/bin/sparkler.sh inject -id j1 -su http://<yoursite>.com
build/bin/sparkler.sh crawl -id j1

If the above commands deoesn’t make sense watch the video below.

Making changes to code

Import the project to your IDE and edit the source code. Then follow build instructions in the above.

Adding a new plugin

Contributing to Project Source Code

If you fix bugs or add new features, please raise a pull request on github. Learn more about github pull requests here

Contact developers on slack if you have questions.