- AIL framework 5.3 released with chat explorer, Discord and Telegram monitoring, automatic translation, new features and various bugs fixed
- Streamlining AIL & Lacus Deployment: An Automated Installer for LXD
- AIL framework 5.2 released with new features and various bugs fixed
- AIL framework 5.1 released with new features and many bugs fixed
- AIL Framework version 5.0 released - Major Rewrite, Kvrocks Database, and Lacus Crawler Migration
- AIL Framework version 4.2 released including typo squatting tracker, improved AIL2AIL sync, zerobinz fetcher and many bugs fixed
- AIL Framework version 4.1 released with a new investigation/case handling, improved MISP export and many improvements
- Installing AIL with lxc/lxd
- AIL Framework version 4.0 released with new synchronisation feature, tracker webhook improvement and many bugs fixed.
- AIL Framework version 3.7 released with many bugs fixed, improvement and new feeders.
- All posts ...
Streamlining AIL & Lacus Deployment: An Automated Installer for LXD
Feb 12, 2024 • Niclas Dauster
Streamlining AIL & Lacus Deployment: An Automated Installer for LXD
The AIL framework (Analysis of Information Leak) has long been at the forefront of privacy and data leak analysis, providing tools for monitoring, analyzing, and managing sensitive unstructured information. The introduction of a dedicated LXD installer marks a significant milestone in making AIL more accessible and easier to deploy, especially in combination with Lacus, a capturing system using instrumented browser capabilities. This development is particularly pertinent within the scope of the MISP-LEA Project, as it underscores a strategic move towards enhancing the accessibility and deployment efficiency of cybersecurity tools for analyzing information leaks, gathering intelligence or Tor hidden services.
LXD, a next-generation system container and virtual machine manager, offers an efficient and secure way to run multiple isolated Linux systems (containers) on a single host. Leveraging LXD for AIL and Lacus environments means streamlined deployments and better resource utilization.
Key Features of the LXD Installer
The LXD installer for AIL and Lacus brings several key benefits to the table:
Simplified Installation Process: The LXD installer for AIL and Lacus simplifies the setup process, making it accessible even to those with minimal experience in container management.
Resource Efficiency: LXD’s lightweight nature means that AIL and Lacus can run on a wide range of hardware setups without significant resource overhead, making it ideal for both small-scale investigations and large-scale deployments.
The LXD installer for AIL and Lacus is designed to be user-friendly and straightforward. Here’s a high-level overview of the installation process. For detailed instructions, refer to the LXD installer README.
- Prerequisites: Ensure that LXD is installed on the host system. If not, follow the official LXD installation guide. Additionally, ensure that
jqis installed on the host system.
- Clone the Repository: Clone the AIL framework repository to the host system.
- Run the Installer: Navigate to the
other_installers/LXDdirectory within the AIL framework repository and run the
INSTALL.shscript. Running the script in interactive mode will guide you through the installation process, prompting you for necessary information along the way:
bash INSTALL.sh --interactive
The installation process is comprehensive, as it involves a complete setup of AIL within the container, which can be time-consuming. To expedite this process, you have the option to download pre-built LXD images from the AIL images website. Utilizing these images can significantly reduce installation time by following these steps:
- Import images to LXD
lxc image import <path_to_image> --alias <image_alias>
- Launch containers using the imported images
lxc launch <image_alias> <container_name>
- Get default credentials for AIL
lxc exec <container_name> -- bash -c "grep '^password=' /home/ail/ail-framework/DEFAULT_PASSWORD | cut -d'=' -f2"
Once the installation concludes, the AIL and Lacus containers will be operational and poised for configuration. Verify their status by execute
lxc list. To integrate AIL with Lacus for crawling tasks, access the AIL web interface on
https://<ip_of_ail_container>:7000, navigate to
Crawlers -> Settings, and specify the Lacus Server URL with the IP address of your Lacus container. Here, you also have the opportunity to adjust the number of crawlers to deploy according to your needs. Upon successful configuration, initiating a crawler test should yield a confirmation message, “It works!”.
Monitoring and Management
The LXD CLI allows for comprehensive monitoring and management of your containers. Check their status, access shell environments, and adjust resources as needed. For detailed management techniques, the official LXD documentation is an excellent resource.
Within the container, AIL operates multiple services that can be actively monitored and managed through screen sessions. To view all active screen sessions associated with AIL, execute the following command:
lxc exec <ail_container_name> -- sudo -u ail -- screen -ls
To engage with a specific session, you can connect by running:
lxc exec <ail_container_name> -- sudo -u ail -- screen -r <session_id>
This approach allows for real-time interaction with AIL’s various services, facilitating immediate adjustments or monitoring as needed.
Use Cases and Applications
The development of the LXD installer for AIL, particularly under the MISP-LEA project, was primarily aimed at providing Law Enforcement Agencies with a streamlined and secure method to deploy AIL. Deploying AIL and Lacus within LXD containers brings significant benefits to cybersecurity operations, highlighted by several key applications and use cases:
Environment Replication for Incident Response Training: LXD’s cloning features enable precise replication of operational environments for incident response drills. This hands-on training is crucial for teams to develop effective response strategies, enhancing their preparedness for real-world incidents.
Scalable Deployment for Data Analysis: LXD’s lightweight containers allow for scalable deployment, enabling comprehensive monitoring and analysis across extensive data sets. This scalability is essential for organizations needing real-time analysis of multiple data sources, ensuring thorough coverage without extensive resource requirements.
Rapid Prototyping and Testing: The efficiency of LXD containers supports quick deployment and testing of new security configurations and tools. This rapid prototyping capability is invaluable for staying ahead of emerging threats, allowing for the development and testing of new security measures in a secure environment.
The new LXD installer significantly lowers the barrier to entry for deploying the AIL framework and Lacus, making advanced leak analysis tools accessible to a broader audience. By leveraging the strengths of LXD, this installer ensures a secure, efficient, and user-friendly deployment process.
This overview provides a glimpse into the benefits and capabilities of the new LXD installer for AIL and Lacus. For those interested in diving deeper, visiting the AIL project’s GitHub repository is the best way to start exploring the potential of this powerful tool.
MISP-LEA, a collaborative endeavor between Shadowserver and CIRCL, is a 24-month initiative funded by the European Union. The project’s central aim is to establish operational and enduring MISP and AIL instances dedicated specifically to law enforcement agencies.