AIL Framework 5.5 Released: New OCR Module for Images, Report Generator for Tracker Module, and Numerous Improvements.

Apr 25, 2024 • Team CIRCL

Changes and New Features

  • Documentation: Improved including overview diagrams.
  • OCR Features:
    • Introduced the OCR object and added functionality to retrieve OCRs by daterange with fixes.
    • Added OCR Extractor module and implemented a filter for GIF images in the reprocess tool.
    • Enhanced OCR capabilities with cache addition, correlation between OCR and chat messages, and enabled the OCR extractor by default.
    • Added language detection and translation, a new view for OCR results, and a blueprint for additional languages.
    • Developed a model to get languages, group extracted content by line, process OCR objects, and retrieve all associated images.
  • Performance Enhancements: Reduced memory usage.
  • Domain-Specific Updates:
    • Fixed the last check in card template domain.
  • Tracking and Reporting:
    • Introduced an experimental report generator in the tracker module.
    • Implemented heatmap visualization for the number of user messages.
    • Enhanced message module to show tracker and module matches.
    • Updated trackers UI to allow removal of objects.
  • User Account Management:
    • Added display features for chats and subchannels.

Fixes

  • OCR and UI Improvements:
    • Fixed an issue where ‘None’ was copied incorrectly in date fields.
    • Resolved UI correlation issues in the language block.
  • Correlation Fixes:
    • Multiple fixes applied to the objects selector to enhance functionality.
  • Export and Logging:
    • Fixed export functionality in TheHive component.
  • UI Enhancements:
    • Resolved issues with empty matches and added handling for overlapping matches in the UI matches extractor.
  • Language Processing:
    • Fixed the minimum probability setting for item languages to improve accuracy.

Overview of AIL features

Overview of the AIL framework features

The new AIL OCR module in action

AIL Project OCR module in action

A sample correlation between chat users

AIL chat users correlation

JTAN

Development of the AIL framework is co-funded by the European Union CEF program and CIRCL.

The Action will establish a Joint Threat Analysis Network, an open collaboration group of European computer security incident response teams (CSIRTs) with the focus on collecting, sharing and analysis of technical, operational and strategic threat intelligence. The purpose of this collaboration is to combine unique advantages of different teams to obtain comprehensive situational awareness and actionable information to effectively defend constituencies in each Member State, from critical infrastructure operators targeted by state-sponsored actors to individual citizens affected by cybercrime. The main part of the Action addresses gaps in the Cyber Threat Intelligence (CTI) tooling that is currently used by the national level CSIRTs in Europe. By strengthening individual tools and interconnecting them, the beneficiaries will achieve a new level of common situational awareness and they will benefit from shared knowledge and tooling.