Senior Data Engineering / AI Specialist (Contractor)

Що робити

• Build the batch ingestion pipeline as a resumable, partitioned campaign: source inventory, fetch, classification, de-duplication, a manifest tracking every document, retries and quality gates before publishing.
• Process each document class: native text extraction, GPU-based OCR and layout parsing for scans, tables and drawing title blocks, with escalation of hard pages to stronger models.
• Generate the AI assets: structure-aware chunks, embeddings, hierarchical summaries (document, project, portfolio), extracted identifiers and metadata with provenance, in a versioned asset lake (e.g. S3 with Iceberg).
• Design the indices and the links between them (hybrid BM25 + vector indices partitioned by project, summary and identifier indices) and the retrieval pipeline: pre-filtering, hybrid search, reranking, hierarchical navigation, context compression.
• Own retrieval quality and cost: evaluation harness with a golden question set; benchmarks of parsers, embeddings and indices; cost per 1,000 pages and per question.