Overview
What It Does
Spark Optimization packages a focused coding & development workflow for an AI agent. Optimize Apache Spark jobs with partitioning, caching, shuffle optimization, and memory tuning. Use when improving Spark performance, debugging slow jobs, or scaling data processing pipelines. It is best suited to users who can review the resulting actions and provide only the accounts, files, or command access needed for the task. It is not a substitute for human approval on destructive, financial, security-sensitive, or public-facing actions.
Task ideas
Popular Use Cases
- Review and improve a code change
- Automate a repeatable development task
- Investigate failures and prepare a fix
- Work with repositories and developer tooling
Installation
Install this Agent Skill
Claude Code
npx skills add https://github.com/wshobson/agents --skill spark-optimizationCommands derived from the public GitHub SKILL.md record. Checked 2026-09-02. Review the source before running them.
Before you start
Requirements
| Claude Code | Required / review |
| Codex | Required / review |
| Public SKILL.md source | Required / review |
| Review instructions and requested permissions before installation | Required / review |
| Paid service | Check source |
| Supported system | Check source |
Popularity context
Why It’s Popular
Spark Optimization is a verified Agent Skill from wshobson with a public SKILL.md, compatible with Claude Code, Codex.
Alternatives
