[Hiring] Lead Data Engineer – Palantir, PySpark at Arthur Grand Technologies

Arthur Grand Technologies (www.arthurgrand.com) is in the business of providing staffing and technology consulting services. We have doubled our revenue year over year for the past 5 years. This speaks to the long-lasting relationship and customer satisfaction that we have built in this short span of time. Our company is managed by a team of professionals who worked for big 5 consulting firms for 20+ years.

We are a minority owned staff augmentation and technology consulting company
To keep our valued employees, we need to keep them engaged in challenging, interesting work, offer market-relevant benefits and provide continued opportunities for professional growth.

Job Title:  Lead Data Engineer – Palantir & PySpark

Location: Remote

Job Summary:

We are seeking a highly skilled Data Engineer with hands-on experience in Palantir (Foundry preferred), PySpark, and exposure to reinsurance or insurance data environments. The ideal candidate will play a key role in building scalable data pipelines, optimizing ETL workflows, and enabling advanced analytics and reporting capabilities. This role requires a strong technical foundation in data engineering combined with an understanding of the reinsurance business domain.

Key Responsibilities:

• Design, develop, and maintain data pipelines and ETL workflows using PySpark, SQL, and Palantir Foundry.
• Collaborate with data architects, business analysts, and actuarial teams to understand reinsurance data models and transform complex datasets into usable formats.
• Build and optimize data ingestion, transformation, and validation processes to support analytical and reporting use cases.
• Work within the Palantir Foundry platform to design robust workflows, manage datasets, and ensure efficient data lineage and governance.
• Ensure data security, compliance, and governance in line with industry and client standards.
• Identify opportunities for automation and process improvement across data systems and integrations.

Required Skills & Qualifications:

• 6–10 years of overall experience in data engineering roles.
• Strong hands-on expertise in PySpark (dataframes, RDDs, performance optimization).
• Proven experience working with Palantir Foundry or similar data integration platforms.
• Good understanding of reinsurance including exposure, claims, and policy data structures.
• Proficiency in SQL, Python, and working with large datasets in distributed environments.
• Experience with cloud platforms (AWS, Azure, or GCP) and related data services (e.g., S3, Snowflake, Databricks).
• Knowledge of data modeling, metadata management, and data governance frameworks.
• Familiarity with CI/CD pipelines, version control (Git), and Agile delivery methodologies.

Preferred Skills:

• Experience with data warehousing and reporting modernization projects in the reinsurance domain.
• Exposure to Palantir ontology design and data operationalization.
• Working knowledge of APIs, REST services, and event-driven architecture.
• Understanding of actuarial data flows, submission processes, and underwriting analytics is a plus.

Thanks,

Afrah Faiza

Arthur Grand Technologies Inc

Arthur Grand Technologies is an Equal Opportunity Employer (including disability/vets)

All your information will be kept confidential according to EEO guidelines.