Watch the original stream (in Russian) from Vsevolod Vikulin here – more details, examples and interesting Q&A.
Disclaimer: The following notes are an AI-generated summary based on the transcript of the video
-
Shift from Science to Engineering LLM work today is applied engineering, not data science. Success depends on architecture, component assembly, and practical “hacks,” not on research experiments.
-
Define Terms Before Building Every meeting starts by agreeing on what “LLM,” “RAG,” or “Agent” means. Misaligned definitions cause more project failures than model performance itself.
-
Data Quality Is Everything Clean data reduces model size requirements and drives 95–99% of success. Most real-world data is messy — screenshots in Word files or PDFs make automation nearly impossible.
-
The “On-Prem” Trap Hosting models internally sounds attractive but brings high cost, latency issues, and operational pain. For most companies, API access is cheaper and more sustainable.
-
Fine-tuning Is Rarely Worth It Few LoRA or SFT projects reach production. Better results usually come from smart context design or using stronger base models rather than custom training.
-
The New Role: Context Engineer “Prompt engineer” is outdated. The key skill is identifying which text fragments matter and structuring retrieval — designing the informational plumbing, not just writing clever prompts.
-
Real Constraint: Requirements & Metrics Projects break when basic non-functional specs (response time, output length, user load) are undefined. Engineering discipline, not ML theory, determines success.
-
LLM Hype Meets Corporate Reality 80–90% of presales now involve LLMs, often driven by FOMO. Many clients ask for “multi-agent systems” without understanding the purpose or cost of implementation.
-
ROI Remains Elusive Measuring gains from “employee productivity” automation is nearly impossible. Most ROI calculations depend on consulting guesses, not real efficiency data.
-
Next 5 Years: Gradual Maturity Expect working agents within a year, scalable deployments in 2–3, and standardized LLM tools in about 5 years — typical tech diffusion, not overnight disruption.
-
Architects Are the New Specialists The industry now needs solution engineers who bridge business goals and LLM architectures — professionals who think in workflows, not model types.