Why Did I Build This?
"Relying on ad-ridden, heavily rate-limited third-party websites to archive media from X and YouTube disrupts workflow efficiency. I engineered this isolated bot architecture to directly orchestrate CLI tools (yt-dlp, FFmpeg) as asynchronous subprocesses over Telegram, integrating a strict authorization layer to prevent abuse and ensure absolute ownership of the media pipeline."
Architecture & Decisions
The core operates on the teloxide framework utilizing the tokio asynchronous runtime. While network I/O is handled asynchronously, the actual video extraction is offloaded to yt-dlp via tokio::process::Command. The subprocess's stdout is piped and parsed in real-time using Regex to project a dynamic progress bar onto the Telegram UI. If the downloaded payload exceeds predefined environment size constraints, ffmpeg is conditionally invoked to aggressively compress the media utilizing libx264 and aac codecs with faststart headers. In-memory state (users and audit logs) is strictly protected via Arc<RwLock>, while physical persistence is guaranteed through an atomic I/O write-and-rename mechanism to completely eliminate JSON file corruption.
Key Features
- 01.Real-time asynchronous progress reporting achieved by regex-parsing yt-dlp piped stdout buffers
- 02.Autonomous FFmpeg H.264/AAC downsampling triggered by environment-based file size thresholds
- 03.Native compatibility with Local Telegram API servers via custom endpoint routing to bypass default 50MB upload restrictions
- 04.Atomic file writing (rename syscall over .tmp) to enforce zero-corruption data persistence
- 05.Thread-safe authorization and metric tracking utilizing RwLock synchronization primitives
This project effectively demonstrates the robust integration of the Rust asynchronous ecosystem with native OS subprocesses. Pinging the tokio::process::Command stdout buffer in a non-blocking loop to update a UI progress bar requires precise stream handling. Additionally, to avoid the overhead of a dedicated database, I opted for local JSON storage. To counter the inherent risks of concurrent file I/O corruption, I implemented a strict atomic write pattern leveraging temporary files and OS-level atomic rename operations, guaranteeing absolute data integrity.