>_

Testing Strategy

Ensure your skill works reliably with a systematic testing approach.

The Testing Matrix

Test your skill across three dimensions:

1. Normal Operation

Does the skill work for typical use cases?

Test scenarios:

  • Standard requests within scope
  • Common variations of requests
  • Multiple related requests in sequence

Example for a "React Guidelines" skill:

"Create a button component" "Add click handler with loading state" "Make it accessible"

2. Edge Cases

Does the skill handle unusual but valid requests?

Test scenarios:

  • Boundary conditions
  • Unusual combinations
  • Minimal and maximal inputs

Example:

"Create an empty component" "Create a component with 20 props" "Nested components 5 levels deep"

3. Out of Scope

Does the skill correctly defer or decline?

Test scenarios:

  • Requests clearly outside the skill's domain
  • Requests that seem related but aren't covered
  • Requests that might conflict with the skill

Example for a "React Guidelines" skill:

"How do I set up a database?" "Write a Python script" "Should I use Vue instead?"

Testing Checklist

Before Release

  • [ ] Tested all documented guidelines
  • [ ] Verified examples produce expected output
  • [ ] Checked edge cases don't cause errors
  • [ ] Confirmed out-of-scope handling is appropriate
  • [ ] Tested on all supported platforms

After Release

  • [ ] Monitor user feedback
  • [ ] Test with real-world scenarios
  • [ ] Check for conflicts with popular skills
  • [ ] Verify updates don't break existing behavior

Platform-Specific Testing

Claude Code

bash
# Install the skill
claude skill add your-username/skill-name

# Test in a project
claude "Your test prompt here"

# Verify behavior
# Check that responses follow your guidelines

Codex CLI

bash
# Add the skill
codex config add-skill your-username/skill-name

# Test
codex "Your test prompt here"

ChatGPT

  1. Add skill content to Custom Instructions
  2. Start a new conversation
  3. Test prompts and verify responses

Automated Testing

For thorough testing, create a test suite:

typescript
// skill-tests.ts
interface TestCase {
  prompt: string;
  expectedBehavior: string[];
  shouldNotContain?: string[];
}

const testCases: TestCase[] = [
  {
    prompt: "Create a React button component",
    expectedBehavior: [
      "Uses functional component",
      "Includes TypeScript types",
      "Has accessibility attributes",
    ],
    shouldNotContain: [
      "class component",
      "any type",
    ],
  },
  // Add more test cases...
];

Regression Testing

When updating your skill:

  1. Document current behavior - Save examples of current output
  2. Make changes - Update the skill
  3. Compare outputs - Check for unexpected changes
  4. Verify improvements - Confirm intended changes work

User Testing

Before wide release:

  1. Alpha testing - Test with 2-3 trusted users
  2. Collect feedback - What works? What's confusing?
  3. Iterate - Make improvements
  4. Beta testing - Test with a larger group
  5. Release - Publish to the marketplace

Debugging Common Issues

Skill Not Activating

Check:

  • Is the skill installed correctly?
  • Is the prompt relevant to the skill's domain?
  • Are there conflicting skills with higher priority?

Inconsistent Behavior

Check:

  • Are guidelines specific enough?
  • Are there contradicting rules?
  • Is the skill too broad?

Unexpected Outputs

Check:

  • Are examples clear?
  • Is the context section accurate?
  • Are there edge cases not covered?

Next Steps

Edit this page on GitHub