Senior AI Software Engineer

Type of Requisition:

Pipeline

Clearance Level Must Currently Possess:

Secret

Clearance Level Must Be Able to Obtain:

Top Secret

Public Trust/Other Required:

None

Job Family:

Data Science and Data Engineering

Job Qualifications:

Skills:

Agile Software Development, Back-End Development, Microservices Architecture

Certifications:

None

Experience:

7 + years of related experience

US Citizenship Required:

Yes

Job Description:

GDIT is seeking a highly motivated and experienced Senior AI Software Engineer to support the development, modernization, integration, and sustainment of our on-premises, AI-powered enterprise chatbot platform supporting the U.S. Department of State in Sterling, VA.

The selected candidate will design, develop, and integrate software solutions leveraging Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), vector databases, document processing technologies, and modern AI inference frameworks.

This role requires a strong foundation in software engineering, Python development, API integration, containerization, and AI application architecture. The engineer will help transition existing AI services toward a scalable, containerized architecture while introducing new capabilities that enhance system performance, reliability, security, and user experience.

The ideal candidate is a hands-on software engineer who can work across the AI application stack, from backend development and model serving integration to data processing, automated testing, and deployment.

The individual will collaborate with AI/ML engineers, data engineers, infrastructure specialists, cybersecurity teams, and government stakeholders to deliver secure, reliable AI capabilities in highly controlled, classified, and air-gapped environments.

Key Responsibilities

1. AI Application Development & Integration

• Design, develop, test, and maintain Python-based applications, microservices, and APIs supporting an on prem chatbot.
• Extend, configure, and integrate OpenWebUI with enterprise AI services and backend systems.
• Develop custom OpenWebUI Functions, Tools, and integrations to support mission-specific use cases.
• Integrate approved LLMs into chatbot using OpenAI-compatible APIs and inference frameworks such as vLLM and Ollama.
• Develop reusable software components for AI model access, prompt management, response processing, and application workflows.
• Collaborate with AI/ML engineers to improve model inference performance, response quality, and application reliability.
• Support multimodal AI use cases, including document understanding, optical character recognition, and vision-language model integration.
• Develop backend capabilities to support secure enterprise AI workflows.

2. Retrieval-Augmented Generation (RAG) & Document Processing

• Design, implement, and optimize RAG workflows for enterprise document search and question answering.
• Integrate document processing technologies such as Docling, Tesseract, and vision-language models into automated ingestion pipelines.
• Develop document chunking, metadata extraction, embedding generation, and retrieval workflows.
• Integrate embedding models and reranking models to improve retrieval accuracy and relevance.
• Work with PostgreSQL and pgvector to support vector search, semantic retrieval, and document indexing.
• Develop APIs and background processing services for document ingestion and knowledge base management.
• Collaborate with data engineers to ensure accurate storage, indexing, and retrieval of enterprise information.
• Implement testing and evaluation approaches to measure RAG accuracy, retrieval relevance, and answer quality.

3. AI Model Serving & Inference Integration

• Integrate and maintain AI inference services using vLLM, Ollama, and compatible model-serving frameworks.
• Support the deployment and configuration of approved open-weight LLMs, embedding models, rerankers, and vision-language models.
• Develop software integrations using OpenAI-compatible REST APIs.
• Assist in optimizing inference configurations, including context length, GPU memory utilization, concurrency, and model serving performance.
• Support the transition from existing inference frameworks to improved architectures as technologies mature.
• Collaborate with AI/ML engineers to troubleshoot inference errors, model loading issues, and integration failures.
• Develop benchmarking and performance-testing utilities for AI workloads.

4. Containerization & Platform Modernization

• Support the migration of DS Chat components from traditional deployments into containerized services.
• Develop, configure, and maintain container images using Podman or Docker.
• Create and maintain Podman Compose configurations defining application services, networking, volumes, and environment variables.
• Integrate services such as OpenWebUI, vLLM, Docling, PostgreSQL, pgvector, PgBouncer, and Redis.
• Develop scripts and automation for repeatable installation, deployment, upgrades, and recovery.
• Implement health checks, readiness checks, logging, and service dependency management.
• Troubleshoot container networking, authentication, storage, GPU access, and service connectivity.
• Collaborate with infrastructure and cybersecurity teams to improve deployment consistency and security.

5. Agentic AI & Future Capabilities

• Research, prototype, and develop agentic AI capabilities for future DS Chat releases.
• Implement controlled tool-calling and multi-step AI workflows.
• Develop integrations that allow approved AI models to interact with authorized enterprise APIs and services.
• Explore emerging technologies involving AI orchestration, workflow automation, and Model Context Protocol (MCP).
• Design secure execution boundaries, permissions, audit logging, and approval mechanisms for AI-driven actions.
• Collaborate with stakeholders to evaluate new AI capabilities for mission relevance, security, and operational feasibility.
• Implement safeguards to support human oversight and prevent unauthorized actions by AI-enabled systems.

6. Software Engineering, Testing & DevSecOps

• Apply software engineering best practices, including modular architecture, code reviews, documentation, and version control.
• Develop automated unit, integration, regression, and performance tests using frameworks such as pytest.
• Build integration tests validating communication between OpenWebUI, inference services, document processing components, and databases.
• Support CI/CD processes appropriate for disconnected and classified environments.
• Troubleshoot production issues through application logs, metrics, traces, and system diagnostics.
• Integrate operational logging and monitoring capabilities with tools such as Splunk.
• Assist with vulnerability remediation, dependency management, software supply chain reviews, and container security assessments.
• Produce technical documentation, deployment procedures, architecture diagrams, and operational runbooks.
• Participate in Agile development activities, including sprint planning, technical design reviews, and demonstrations.

7. Security & Classified Environment Support

• Design software solutions suitable for secure, on-premises, air-gapped government environments.
• Implement secure authentication integration, role-based access controls, API authorization, and data protection measures.
• Ensure AI services communicate only through authorized internal network paths.
• Support software packaging and offline deployment using preapproved dependencies, container images, and model artifacts.
• Collaborate with cybersecurity personnel to address security findings and support Risk Management Framework (RMF) activities.
• Assist with technical documentation and evidence required for TRB/CCB reviews and Authority to Operate (ATO) processes.
• Follow approved procedures for handling sensitive and classified information.

Required Qualifications

• Minimum Secret Clearance, no sponsorship
• Bachelor's degree in Computer Science, Software Engineering, Artificial Intelligence, Computer Engineering, or a related technical field, or equivalent practical experience.
• 5+ years of professional software engineering experience, including backend application development.
• Strong proficiency in Python and experience developing production-quality software.
• Experience developing RESTful APIs and integrating distributed application services.
• Hands-on experience integrating LLMs or generative AI capabilities into software applications.
• Understanding of transformer-based models, inference APIs, prompt engineering, and LLM application development.
• Experience with RAG concepts, including document ingestion, embeddings, vector search, and retrieval pipelines.
• Experience working with relational databases such as PostgreSQL.<