Senior Software Security Engineer - AI Benchmark Development
Gramian Consulting
Job description
About the role
Gramian Consultancy is looking for an experienced Senior Software Security Engineer to develop high-quality training and evaluation tasks for advanced AI coding models. The role turns real-world software vulnerabilities (CVEs, security advisories, open-source bug reports) into realistic Harbor-format audit-and-patch tasks across several programming languages. It combines software security research, vulnerability reproduction, benchmark development, automated testing and technical review, and is fully remote.
The engagement is a 3-month contractor assignment starting next week, at 40 hours per week.
Key responsibilities
- Source and investigate real-world vulnerabilities from CVEs, advisories, bug reports and open-source repositories
- Classify vulnerabilities by the most specific applicable CWE and map them to relevant OWASP Top 10 categories
- Reproduce security issues and build proof-of-vulnerability tests
- Build Harbor-format audit-and-patch tasks and write task instructions that hide vulnerable code, exploits and expected fixes
- Create reproducible Docker-based environments with pinned dependencies and clear setup instructions
- Design and harden automated verifiers against gaming, test manipulation and hardcoded solutions
- Review and score AI-generated patches and reasoning with structured rubrics, and flag tasks solvable by pattern matching or copying upstream patches
- Document vulnerability context, reproduction steps, expected behavior, patch requirements, test methodology and review decisions
Required profile
- 5+ years of professional software engineering experience
- Deep practical knowledge of application security and ability to investigate vulnerabilities independently
- Ability to work independently in a fully remote, research-oriented environment
Required skills
- Expert-level proficiency in at least one of: Python, JavaScript/TypeScript, Java, C/C++, C#, Go, Rust, Ruby, PHP, Kotlin, Swift
- Understanding of CWE, CVE and OWASP Top 10; hands-on security testing tools such as sanitizers, fuzzers, CodeQL and Semgrep
- Strong Git, Linux, Bash and Docker skills
- Ability to write clear technical documentation
Questions fréquentes
Why are you reporting this job?
Explore further
Salaries, guides and searches in Pakistan.
Salaries by job title
Apply in 30 seconds
Enter your email to apply. An account will be created automatically.
By continuing, you accept our terms of use.
Already have an account? Login
A question about this job?
Ask it here: you will get the full job summary by e-mail, right away.
Published 6 hours ago
Expires 1 month from now
3 views · 0 interested
Boost your chances
Upload your CV — we will match you with relevant openings.
Analyzing your CV...
Gramian Consulting