AIL Project features
Threat Intelligence Platform
AIL Project features
AIL helps analysts discover, correlate, and monitor valuable intelligence from unstructured and structured sources at scale.
Screenshots
Core features
- Modular architecture to handle streams of unstructured or structured information
- Default support for external ZMQ feeds, such as provided by CIRCL or other providers
- Multiple feed support
- Each module can process and reprocess the information already processed by AIL
- Detecting and extracting URLs including their geographical location (e.g. IP address location)
- Extracting and validating potential leaks of credit card numbers, credentials, and other sensitive data
- Extracting and validating leaked email addresses, including DNS MX validation
- Module for extracting Tor .onion addresses (to be further processed for analysis)
- Keep tracks of duplicates (and diffing between each duplicate found)
- Extracting and validating potential hostnames (e.g. to feed Passive DNS systems)
- A full-text indexer module to index unstructured information
- Statistics on modules and web
- Real-time modules manager in terminal
- Global sentiment analysis for each providers based on nltk vader module
- Terms, set of terms, regex and YARA tracking and occurrence
- Extended tracker to find potential typo-squatting domains using ail-typo-squatting
- Many more modules for extracting phone numbers, credentials and others
- Alerting to MISP to share found leaks within a threat intelligence platform using MISP standard
- Detect and decode encoded files (Base64, hex encoded or custom decoding schemes) and store files
- Detect Amazon AWS and Google API keys
- Detect Bitcoin address and Bitcoin private keys and correlation with additional Bitcoin addresses
- Detect private keys, certificates and keys (including SSH and OpenVPN)
- Detect IBAN bank accounts
- Tagging system with MISP Galaxy and MISP Taxonomies tags
- UI paste submission
- Create events on MISP and cases on The Hive
- Automatic paste export at detection on MISP (events) and The Hive (alerts) on selected tags
- Extracted and decoded files can be searched by date range, file type (mime-type) and encoding discovered
- Graph relationships between decoded files (hashes), similar PGP UIDs and addresses of cryptocurrencies
- Tor hidden services crawler to crawl and parse output
- Tor onion availability is monitored to detect up and down of hidden services
- Browser hidden services are captured and integrated in the analysed output, including a blurring screenshot interface
- Tor hidden services are part of the standard framework, all AIL modules are available to crawled hidden services
- Generic web crawler to trigger crawling on demand or at regular interval URL or Tor hidden services
- An advanced detection engine per keyword supporting words, sets, regular expressions and YARA rules
- YARA retro-hunting to perform searches on collected items
- AIL synchronisation features to sync with other AIL instances
Recent features introduced
- Unified search interface with best-match/most-recent sorting and integrated date filtering
- Chat Explorer with Discord, Telegram, and Matrix ingestion support, including replies, forwarded metadata, threads, subchannels, and message relationships
- Integrated translation workflows for chats and multilingual analysis improvements
- Built-in YARA editor (CodeMirror) for creating and managing rules directly in AIL
- Image and screenshot intelligence improvements, including AI-generated descriptions and better unsafe-content handling/tagging
- Expanded OCR and QR extraction capabilities, including improved correlation and support for inverted QR codes
- Full PDF processing pipeline with metadata extraction, translation support, and PDF-to-Markdown content extraction
- Passive SSH integration to correlate SSH keys with IPs, domains, and onion services for infrastructure analysis/deanonymization
- I2P crawling support (in addition to Tor/web crawling), with improved crawler stability and capture import workflows
- New investigation workflow improvements (investigation statuses, direct-correlation prioritization, and tracker-to-retro-hunt workflow)
- Organization-aware access controls and stronger account security with 2FA (TOTP/HOTP)
- Extended mail and username search capabilities with improved performance and pagination
Processing overview
AIL provides a modular approach with different Python modules in charge of the various features. Modules can be plugged and started live to allow distribution of load accross multi processors.
