From Weakypedia, the free technical encyclopedia
Autonomous CTI Analysis Engine
| Project: SecIntel AI | |
|---|---|
Llama-3 Generative Reasoning Dashboard
|
|
| Developer | Vishal R. |
| Core Model | Llama-3 8B (4-bit) |
| Dataset Size | 172,563 Raw Reports |
| Unique Actors | 90,013 Profiles |
| Stack | Python, Streamlit, Cloudflare |
| Optimization | O(1) Hash-Mapping |
Contents
- Overview
- Methodology
- Performance
- External Resources
The Autonomous CTI Analysis Engine is a high-performance cybersecurity framework designed to automate Tier-1 SOC (Security Operations Center) triage. By leveraging Large Language Models (LLMs) and deterministic data refinement, it consolidates fragmented threat intelligence from sources like VirusTotal and AbuseIPDB into a unified, actionable index.
Methodology
The engine employs a multi-stage pipeline to handle the "Data Avalanche":
- Heuristic Refinement: Distills raw JSON logs by 85%, removing redundant metadata.
- Schema Normalization: Converts disparate data formats into a single, high-fidelity master object.
- Generative Reasoning: Uses Llama-3 to interpret complex threat scores and provide human-readable mitigation strategies.
Performance Metrics
Moving from legacy sequential file scanning to an optimized hash-mapped index resulted in a 100,000x speed increase in data retrieval:
- Legacy Search Time: ~120 Seconds per IP
- Optimized Search Time: < 0.001 Seconds
External Resources & Documentation
- Source Code: GitHub Repository
- Frontend Design: SecIntel Platform Design
- Research Datasets: Google Drive Datasets