Install Lookyloo

Requires Python 3.8 or higher, expects Ubuntu 20.04 or more recent.

Install Lookyloo Dependencies

Install Poetry

Poetry is tool to handle dependency installation as well as building and packaging of Python packages.

curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python3

More details in the the installation guide.

Install Splash

Splash is a javascript rendering service with an HTTP API. It’s a lightweight browser with an HTTP API, implemented in Python 3 using Twisted and QT5.

You will need a running instance, and the preferred way is via Docker.

If you installed Lookyloo on Windows, you already installed docker. You only have to execute the second line of the following snippet
On Ubuntu, you may want to have a look at this installation guide, which will provide a more recent version of docker.

And if you just want to get going on linux, here it is:

sudo apt install docker.io
sudo docker pull scrapinghub/splash

Install Redis

Redis: An open source (BSD licensed), in-memory data structure store, used as a database, cache and message broker.

Redis should be installed from the source, and the repository must be in the same directory as the one you will be cloning Lookyloo in to.

In order to compile and test redis, you will need a few packages:

sudo apt-get update
sudo apt install build-essential tcl
git clone https://github.com/antirez/redis.git
cd redis
git checkout 6.2
make
# Optionally, you can run the tests:
make test
cd ..

Install uwhoisd (Optional)

In order to use the Universal Whois you have to install uwhoisd from Lookyloo repository and the whois package:

sudo apt-get update
sudo apt-get install whois
git clone https://github.com/Lookyloo/uwhoisd
cd uwhoisd
poetry install

Then configure the service file

nano etc/systemd/system/uwhoisd.service.sample
sudo cp etc/systemd/system/uwhoisd.service.sample /etc/systemd/system/uwhoisd.service
cd ..
sudo systemctl daemon-reload
sudo systemctl start uwhoisd
sudo systemctl status uwhoisd

If everything is ok, activate the service for reboot

sudo systemctl enable uwhoisd

Install Lookyloo

Prerequisites

  • Poetry is installed.

  • Redis is installed from the source. The repository must be in the same directory as the one you will be cloning Lookyloo in to.

  • Splash is installed.

Procedure

  1. Clone the Lookyloo repository.

    git clone https://github.com/Lookyloo/lookyloo.git

  2. Change directory

    cd lookyloo

  3. Run the commands:

    poetry install
echo LOOKYLOO_HOME="'`pwd`'" > .env
If you have error messages about python 2.7, it is an issue with poetry, and the quick and dirty fix is to edit ~/.poetry/bin/poetry and add a 3 at the end of the line: 
#!/usr/bin/env python3

instead of

#!/usr/bin/env python

Install pymisp dependencies (optional)

If you want to install the pymisp dependencies, please install libfuzzy-dev and the run poetry with -E option

sudo apt-get install libfuzzy-dev
poetry install -E misp

Configuration

  1. Initialize the user configuration

    cp config/generic.json.sample config/generic.json
cp config/modules.json.sample config/modules.json

  2. Edit the config files accordingly to your needs: see the _notes key in the JSON file. You can also look at the integration page for more information regarding the modules, and the configuration page for the generic configuration options.

  3. Make sure the configuration files are valid, and pull the 3rd party dependencies for the website

    poetry run update --yes

Run Splash

  • From another terminal, run the following command:

sudo docker run -p 8050:8050 -p 5023:5023 scrapinghub/splash --disable-browser-caches
# If you have a lot of ram at hand, you may want to run it this way:
# sudo docker run -p 8050:8050 -p 5023:5023 scrapinghub/splash -s 100 -m 50000 --disable-browser-caches

Note: you may want to remove --disable-browser-caches, as it is causing issue on some websites (generally, splash segfaults).

Run the app

poetry run start

Open the website

Unless you changed the default in config/generic.json, the web interface will be reachable at http://0.0.0.0:5100/