This job offer is not available in your country.

Data Engineer

Axiom Software Solutions LimitedGB

30+ days ago

Job type

Job description

Job title : Data Engineer

Location : UK

Duration : 6-month contract Inside IR35

Working Mode : Fully Remote

Vetting : Active SC Clearance

Job Description :

Data Engineer Skills and Technologies Data Engineering :

Strong understanding of data concepts
data types, data structures, schemas (both JSON and Spark), schema management etc
Strong understanding of complex JSON manipulation - Experience working with Data Pipelines using a custom Python / PySpark frameworks
Strong understanding of the 4 core Data categories (Reference, Master, Transactional, Freeform) and the implications of each, particularly managing / handling Reference Data.
Strong understanding of Data Security principles
data owners, access controls
row and column level, GDPR etc including experience of handling sensitive datasets
Strong problem solving and analytical skills, particularly able to demonstrate these intuitively

Requirements

Required Skills :

Languages / Frameworks

JSON

YAML

Python (as a programming language, not just able to write basic scripts. Pydantic experience would be a bonus.)

PySpark

Delta Lake

Bash (both CLI usage and scripting)

Markdown

Scala (bonus, not compulsory)

Azure SQL Server as a HIVE Metastore (bonus) Technologies

Azure Databricks

Apache Spark

Delta Tables

Data processing with Python

PowerBI (Integration / Data Ingestion)

JIRA

Data Engineer • GB