Module Reference

Complete reference of all modules, classes, and functions in Capcat.

Modules by Package

Capcat

• capcat.cli - CLI entry point for Capcat
• capcat.commands.add_source - Add-source command - interactive RSS source addition
• capcat.commands.fetch - Batch fetch command - processes multiple sources via the unified processor
• capcat.commands.generate_config - Generate-config command - launches the interactive source config generator
• capcat.commands.init - Implementation of capcat init command
• capcat.commands.remove_source - Remove-source command - interactive source removal with backup/undo support
• capcat.commands.single - Single article fetch command
• capcat.core.article_fetcher - Shared article fetching functionality for Capcat sources
• capcat.core.async_pdf_manager - Asynchronous PDF download manager to prevent thread pool exhaustion
• capcat.core.circuit_breaker - Circuit Breaker pattern implementation for Capcat
• capcat.core.cli_recovery - CLI error recovery and user guidance system
• capcat.core.cli_validation - Enhanced CLI validation and error handling for better user experience
• capcat.core.command_logging - Enhanced command logging for CLI debugging and audit trail
• capcat.core.config - Configuration management for Capcat
• capcat.core.config.source_base - Base configuration classes for news sources
• capcat.core.config.source_registry - Source Registry for managing all available news sources and their configurations
• capcat.core.constants - Application-wide constants for Capcat
• capcat.core.content_sanitizer - Content Sanitizer - Archive isolation for Capcat
• capcat.core.conversion_executor - Shared executor pool for HTML-to-Markdown conversion to prevent nested ThreadPoolExecutor deadlock
• capcat.core.date_extractor - Extract publication dates from HTML pages
• capcat.core.design_system_compiler - Design System Compiler for Capcat HTML Generation

Compiles CSS custom properties from the design system into hardcoded values for performance optimization and self-contained HTML generation

• capcat.core.downloader - Media downloader for Capcat
• capcat.core.enhanced_argparse - Enhanced ArgumentParser with better error messages and validation
• capcat.core.error_handling - Comprehensive error handling and recovery system for Capcat
• capcat.core.ethical_scraping - Ethical scraping utilities for Capcat
• capcat.core.exceptions - Custom exceptions for Capcat application
• capcat.core.formatter - HTML to Markdown converter for Capcat
• capcat.core.html_post_processor - HTML Post-Processor for Capcat Archives Handles post-processing HTML generation after article scraping is complete
• capcat.core.image_processor - Global Image Processor for Capcat
• capcat.core.interactive - Interactive mode for Capcat
• capcat.core.logging_config - Logging configuration for Capcat
• capcat.core.media_config - Media Configuration Manager for different news sources
• capcat.core.media_executor - Shared executor pool for media processing to prevent nested ThreadPoolExecutor deadlock
• capcat.core.media_processor - Media processing component for Capcat
• capcat.core.network_resilience - Network Resilience Patterns for Source Processing

Clean architecture implementation applying SOLID principles:

• Single Responsibility: Each class has one clear purpose
• Open/Closed: Extensible via strategy pattern
• Liskov Substitution: RetryStrategy implementations interchangeable
• Interface Segregation: Small, focused interfaces
• Dependency Inversion: Depend on abstractions not concretions
• capcat.core.pdf_landing_resolver - Resolve direct PDF URLs to their HTML landing pages where possible
• capcat.core.progress - Progress indicators and status reporting for Capcat
• capcat.core.rate_limiter - Rate limiting system for Capcat to prevent overwhelming source servers
• capcat.core.retry - Retry mechanisms with exponential backoff for Capcat
• capcat.core.retry_skip - Retry-and-Skip Logic for Network Resilience

Implements intelligent retry-and-skip mechanism for sources that timeout or refuse connection

• capcat.core.session_pool - Global session pooling for optimal network performance across all sources
• capcat.core.shutdown - Graceful shutdown handling for Capcat
• capcat.core.source_config - Source configuration for optimized URL detection in Capcat
• capcat.core.source_config_mirror - Mirror builtin source configs to userspace Config/sources/active/
• capcat.core.source_configs - Modular source configuration system with backward compatibility
• capcat.core.source_system.add_source_command - Professional implementation of the add-source command using clean architecture principles
• capcat.core.source_system.add_source_service - Service layer for the add-source command
• capcat.core.source_system.base_source - Abstract base class for all news sources
• capcat.core.source_system.bundle_manager
• capcat.core.source_system.bundle_models - Data models for bundle management
• capcat.core.source_system.bundle_service - Service layer for bundle management
• capcat.core.source_system.bundle_ui - User interface components for bundle management
• capcat.core.source_system.bundle_validator - Bundle validation logic
• capcat.core.source_system.config_driven_source - Config-driven source implementation
• capcat.core.source_system.discovery_strategies - Discovery strategy implementations for article discovery
• capcat.core.source_system.enhanced_remove_command - Enhanced remove-source command with advanced features:
• Dry-run mode
• Automatic backups
• Usage analytics
• Batch removal from file
• Undo/restore functionality
• capcat.core.source_system.feed_discovery - RSS/Atom feed discovery utilities
• capcat.core.source_system.feed_parser - Feed parser abstraction for RSS and Atom feeds
• capcat.core.source_system.performance_monitor - Source performance monitoring system for the hybrid architecture
• capcat.core.source_system.questionary_ui - User interface implementation using questionary for interactive prompts
• capcat.core.source_system.removal_ui - User interface implementation for remove-source command
• capcat.core.source_system.remove_source_command - Base classes and implementations for the remove-source command
• capcat.core.source_system.remove_source_service - Service layer for remove-source command
• capcat.core.source_system.rss_feed_introspector
• capcat.core.source_system.source_analytics - Source usage analytics and statistics
• capcat.core.source_system.source_backup_manager - Backup and restore functionality for source configurations
• capcat.core.source_system.source_config - Source configuration system for specialized sources
• capcat.core.source_system.source_config_generator
• capcat.core.source_system.source_factory - Source factory for creating and managing news source instances
• capcat.core.source_system.source_registry - Source registry for automatic discovery and management of news sources
• capcat.core.source_system.validation_engine - Enhanced configuration validation engine for the source system
• capcat.core.storage_manager - Storage management component for Capcat
• capcat.core.streamlined_comment_processor - Streamlined comment processor for optimizing nested structure handling and reducing conversion time
• capcat.core.template_renderer - Simple Template Renderer for Capcat Replaces variables with actual values from configuration
• capcat.core.theme_utils - Theme utilities for hash-based theme persistence
• capcat.core.timeout_config - Adaptive timeout configuration for Capcat
• capcat.core.timeout_wrapper - Timeout wrapper utilities for preventing hanging operations
• capcat.core.tui_context - TUI context flag and per-fetch result accumulation
• capcat.core.unified_article_processor - Unified Article Processor - Universal entry point for all article processing
• capcat.core.unified_media_processor - Unified Media Processor Integration Layer
• capcat.core.unified_source_processor - Unified Source Processor for Capcat
• capcat.core.update_manager - Update Manager for Capcat
• capcat.core.url_utils - URL validation and normalization utilities for Capcat
• capcat.core.utils - Core utilities for the Capcat application
• capcat.htmlgen.factory - Factory for creating ArticleHTMLGenerator instances
• capcat.htmlgen.generator - HTML Generator for Capcat - Static Site Generation Creates self-contained HTML files from markdown content with embedded CSS and JavaScript
• capcat.scripts.generate_source_config - Interactive script to generate comprehensive YAML configuration files for config-driven sources in Capcat
• capcat.sources.builtin.custom.hn.source - Hacker News source implementation using the official Firebase API
• capcat.sources.builtin.custom.lb.source - Lobsters source implementation for the new source system
• capcat.sources.builtin.custom.medium.source - Medium
• capcat.sources.builtin.custom.substack.source - Substack
• capcat.sources.builtin.custom.twitter.source - Twitter/X
• capcat.sources.builtin.custom.vimeo.source - Vimeo specialized source implementation
• capcat.sources.builtin.custom.youtube.source - YouTube specialized source implementation
• capcat.tui - TUI entry point - delegates to core interactive module

Root

• add_jekyll_frontmatter - Add Jekyll front matter to all HTML files so Jekyll processes them
• cleanup_development_files - Remove internal development files from git tracking
• cleanup_repo - Repository cleanup script: Remove unnecessary files from git tracking
• convert_docs_to_html - Convert Markdown documentation to clean HTML with minimal styling
• convert_to_markdown
• delete_h4_colon - Delete colon after </h4> tags
• update_html_includes - Update all HTML files in docs/ to use includes system for header and footer
• update_html_jekyll - Update all HTML files in docs/ to use Jekyll includes for header and footer
• update_includes - Update script: Replace old header/footer HTML with new includes

Scripts

• scripts.doc_generator - Documentation Generator for Capcat

Automatically extracts and generates comprehensive documentation from the codebase

• scripts.generate_diagrams - Generate Architecture Diagrams for Capcat

Creates Mermaid diagrams for system architecture, data flow, and component relationships

• scripts.generate_source_config - Interactive script to generate comprehensive YAML configuration files for config-driven sources in Capcat
• scripts.replace_exhaustive - Replace “Exhaustive” with “Comprehensive” in all website files
• scripts.replace_menus_with_menu - Replace ‘menus’ with ‘menu’ in text under Mermaid diagrams in diagrams/*
• scripts.setup_dependencies - Automated Dependency Setup and Repair Script for Capcat

This script provides robust virtual environment management with:

• Intelligent venv validation and repair
• Dependency verification and installation
• Path corruption detection and fixing
• Fallback mechanisms for common issues
• Comprehensive logging and diagnostics

Usage: python3 scripts/setup_dependencies

Statistics

• Total Modules: 110
• Total Classes: 196
• Total Functions: 316
• Public Functions: 225
• Documentation Coverage: 71.2%