Data Engineer - AI Practice Team

ABS is seeking an exceptional Data Engineer to join us full-time on our Artificial Intelligence (AI) Practice Team. In this role, you will design and operate the data foundations that power AI chat assistants, custom AI models, and AI-driven process optimization for ABS Consulting clients. You will build robust pipelines that integrate structured and unstructured data, standardize and tag enterprise content, and enable scalable, low-latency retrieval for AI workloads. Working closely with AI engineers, consultants, and domain experts, you will turn messy real-world data into production-grade data assets that deliver measurable business impact.

What You Will Do

• Design, build, and maintain scalable ETL/ELT pipelines to ingest, clean, and transform structured and unstructured data for AI assistants and custom models.
• Integrate diverse knowledge repositories (documents, policies, procedures, standards, databases) into centralized data platforms that support retrieval-augmented generation (RAG) and search.
• Implement data standardization, normalization, and tagging pipelines to align content with enterprise taxonomies and ontologies.
• Collaborate with AI/ML engineers to productionize model‑ready datasets, feature stores, and embeddings for prediction, classification, and optimization use cases.
• Optimize data workflows for reliability, cost, and performance across batch and streaming workloads, including monitoring, alerting, and capacity planning.
• Establish and enforce data quality, lineage, and governance practices to ensure trustworthy inputs to AI systems and process‑automation solutions.
• Automate and templatize common data engineering patterns to accelerate delivery across multiple client engagements and industry domains.
• Partner with consultants and business stakeholders to translate process optimization and analytics requirements into robust, maintainable data solutions.

What You Will Need

Education and Experience

• Bachelor’s degree in Computer Science, Data Engineering, Information Systems, or a closely related technical field; Master’s degree preferred.
• 6+ years of professional data engineering experience designing, building, and operating production data solutions.
• Demonstrated experience working in data‑intensive environments (e.g., analytics platforms, AI/ML workloads, large‑scale content repositories, or enterprise data platforms).
• Hands‑on experience delivering solutions on at least one major cloud provider (AWS, Azure, or Google Cloud), including managed data and analytics services.

Knowledge, Skills, and Abilities

• Strong command of SQL and at least one programming language commonly used in data engineering (Python preferred) for building production‑grade data pipelines.
• Hands‑on experience with modern data processing frameworks and platforms (e.g., Spark, Databricks, Snowflake, BigQuery, Synapse, or similar).
• Proficiency with ETL/ELT orchestration tools and workflows (e.g., Airflow, dbt, Azure Data Factory, AWS Glue, or equivalent).
• Experience designing and operating data lakes/lakehouses and integrating multiple data sources (relational, NoSQL, files, APIs) into cohesive data models.
• Deep experience working with unstructured and semi‑structured data (documents, PDFs, JSON, logs), including content extraction, normalization, and metadata/tagging.
• Familiarity with AI/ML data patterns, including feature engineering, embeddings, vector databases, and retrieval‑augmented generation (RAG) pipelines.
• Strong understanding of data modeling, data quality, data governance, and lineage practices for regulated or compliance‑sensitive environments.
• Proficiency with cloud‑native data services (e.g., S3/ADLS/GCS, managed warehouses, streaming services like Kafka/Kinesis/Event Hubs).
• Solid grounding in software engineering best practices (version control, CI/CD, testing, code review) as applied to data engineering.
• Must hold a valid right to work status in the UK.

Reporting Relationships

This role reports to a project manager and does not initially include direct reports.

#J-18808-Ljbffr