RECENT POSTS
- AIL v7.0 - Forums Crawling and Collection in AIL
- AIL v6.8 - Hunting, Graphs & Analyst Experience
- AIL Framework Release: v6.7 - The Improved Search Interface
- AIL v6.6 – Advanced PDF Processing and Manual Crawling Support
- AIL v6.5 – I2P Crawler, Image Descriptions, and Enhanced Search
- AIL v6.4 Release Notes with Mail Search functionality, improved crawling capabilities and many bugs fixes
- AIL v6.3 – Passive SSH Integration for Onion Correlation and Deanonymization
- AIL 6.2: Smarter Analysis, Search and Enhanced User Experience
- The Art of Pivoting - How You Can Discover More from Adversaries with Existing Information
- AIL Project v6.1 released with new features including unsafe filter for Tor crawling, many bugs fixed and Telegram attachment analysis (2025-02-06)
- All posts ...
AIL v7.0 - Forums Crawling and Collection in AIL
Jul 9, 2026 • AIL Project Team
AIL v7.0 introduces a new feature to collect, analyse, extract and correlate forums.
Forums are useful sources of information, but they are often difficult to monitor. Many forums require an account, sessions must be maintained over time, and collectors may face captchas, scraping protection, bans, or other access restrictions. Useful information is also spread across subforums, threads, posts, users, images, reactions, and quoted messages.
With this release, AIL can keep the forum structure and make it easier to analyse. Instead of storing forum pages as simple web captures, AIL now understands forums as dedicated objects:
- Forums
- Subforums
- Threads
- Posts
- User accounts
This makes forum data easier to browse, search, translate, track, correlate, and use in investigations.
Main features
Forum browsing
- Browse forums, subforums, threads, and posts in a dedicated interface.
- Keep the structure of the forum, including breadcrumbs and parent-child relationships.
- View related user accounts, images, reactions, and quoted content.
- Sort threads by last activity.
Forum extraction
- Extract structured data from forum HTML pages.
- Import forums, subforums, threads, posts, users, reactions, images, URLs into AIL.
- Detect languages and translate forum posts.
- Handle post images and user images for analysis and correlation.
Authenticated forum crawling
- Manage forum crawler accounts.
- Support authenticated collection using cookies and local storage.
- Schedule forum, subforum, and thread crawl tasks.
- View crawler status, pending tasks, running tasks, errors, and queues.
Search, tracking, and correlation
- Search forum posts and forum threads.
- Filter search results by forum.
- Use trackers to monitor forum content.
- Correlate forums, subforums, threads, posts, users, images, usernames, and other AIL objects.
- Track usernames across different forums to identify reuse, links, and recurring activity.
Analyst workflows
- View forum user accounts and their activity.
- Browse posts from a specific forum user.
- View activity statistics for forum users.
- Search for specific information inside collected forum content.
- Pivot from a post to its author, thread, subforum, images, reactions, and related objects.
Training/Workshop Online
As our presentation on forum collection and analysis with AIL was not accepted for Underground Economy, we will organise a public presentation to share the feature and demonstrate how it can be used.
Funding
AIL is developed and maintained with the support of the European Union as part of the HOPLITE European Project.
HOPLITE supports the improvement of tools such as AIL and MISP to help law enforcement authorities collect, analyse, and share intelligence.
- More information: https://hoplite-project.eu/
Law enforcement agencies can request access to the operated MISP/AIL-LEA platforms through:
