Dev Environment Setup
Requirements for developing it via docker
- JDK 8 - Install it from https://java.com/en/download/
- Docker - Install it from https://docs.docker.com/engine/installation/
- Maven - Install it from https://maven.apache.org/download.cgi
- An IDE - Get Intellij IDEA Community Edition from here
Docker is a shortcut for quickly launching Solr and admin dashboard using prebuilt image. If you wish to install Solr natively, then skip docker and install solr from http://archive.apache.org/dist/lucene/solr/7.1.0/
Launch Solr and Banana Dashboard
docker run -p 8983:8983 --user sparkler -it uscdatascience/sparkler
Using Solr natively:
Follow instructions highlighted by this URL
Building the Project
Obtaining the Source code for the first time
git clone firstname.lastname@example.org:USCDataScience/sparkler.git cd sparkler
Launch a terminal and
cd to the root of project.
Building whole project:
git pull origin master mvn clean package
The whole project includes API, App, and Plugins. It also runs all the test cases.
To build the core project excluding plugins:
mvn clean package -Pcore
When build task is SUCCESS, it creates a
build directory with the following structure
build/ ┝---bin/ -- Useful scripts | └- sparkler.sh -- Command line interface ┝-- conf/ -- All the config files ┝-- plugins/ -- All the plugin jars └--- sparkler-app*.jar -- Application code except plugins
Running a test crawl
build/bin/sparkler.sh inject -id j1 -su http://<yoursite>.com build/bin/sparkler.sh crawl -id j1
If the above commands deoesn’t make sense watch the video below.
Making changes to code
Import the project to your IDE and edit the source code. Then follow build instructions in the above.
Adding a new plugin
If you fix bugs or add new features, please raise a pull request on github. Learn more about github pull requests here
Contact developers on slack if you have questions.