Capcat requires specific Python packages and system dependencies to function correctly. This document outlines all dependencies, installation procedures, and troubleshooting information.
All Python dependencies are specified in requirements.txt and automatically installed by the wrapper system.
The new wrapper system handles all dependency management automatically:
# Simply run capcat - dependencies are installed automatically
./capcat list sources
# Or use the Python wrapper directly
python3 run_capcat.py list sources
If you prefer manual control over the environment:
# Create virtual environment
python3 -m venv venv
# Activate virtual environment
source venv/bin/activate
# Install dependencies
pip install -r requirements.txt
# Verify installation
python3 -c "import requests, bs4, yaml, markdownify, markdown, pygments; print('All dependencies installed')"
# Install globally (may conflict with other projects)
pip3 install -r requirements.txt
requests # HTTP library for web requests
beautifulsoup4 # HTML parsing and extraction
PyYAML # YAML configuration file support
markdownify # HTML to Markdown conversion
markdown>=3.5.0 # Markdown processing with version constraint
pygments>=2.16.0 # Syntax highlighting with version constraint
run_capcat.py)The Python wrapper handles all dependency-related tasks:
class CapcatWrapper:
def get_python_executable(self):
"""Detects and configures Python environment."""
def create_virtual_environment(self):
"""Creates venv if none exists."""
def install_dependencies(self, python_exe):
"""Installs packages from requirements.txt."""
capcat)Minimal script that delegates to Python wrapper:
#!/bin/bash
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" &>/dev/null && pwd)"
exec python3 "${SCRIPT_DIR}/run_capcat.py" "$@"
# Let wrapper handle it automatically
./capcat list sources
# Or install manually
source venv/bin/activate && pip install -r requirements.txt
brew install python@3.11sudo apt install python3 python3-pip python3-venv# Clean and recreate
rm -rf venv
python3 -m venv venv --clear
# Use different index
pip install -r requirements.txt --index-url https://pypi.org/simple/
# Increase timeout
pip install -r requirements.txt --timeout 120
# Check installed versions
pip list
# Update conflicting packages
pip install --upgrade package_name
# Recreate environment if necessary
rm -rf venv && ./capcat list sources
# Ensure clean virtual environment
python3 -m venv venv --clear
source venv/bin/activate
pip install -r requirements.txt
For development and testing, additional packages may be required:
# Development requirements (not in main requirements.txt)
pip install pytest pytest-cov flake8 black mypy
# Run dependency tests
python3 -c "
import sys
required = ['requests', 'bs4', 'yaml', 'markdownify', 'markdown', 'pygments']
missing = []
for module in required:
try:
__import__(module)
print(f'[OK] {module}')
except ImportError:
missing.append(module)
print(f'✗ {module}')
if missing:
print(f'Missing: {missing}')
sys.exit(1)
else:
print('All dependencies available')
"
requests library is configured with connection pooling:
# In core/config.py
pool_connections: int = 20
pool_maxsize: int = 20
Typical memory usage by dependency:
requests: ~10MB baselinebeautifulsoup4: ~5MB per parsed documentmarkdown: ~3MB for processingpygments: ~8MB for syntax highlighting# Verify package integrity
pip check
# List installed packages and versions
pip freeze
# Audit for security vulnerabilities (requires pip-audit)
pip install pip-audit && pip-audit
requirements.txt in the project root.