Auto-Suggest Coding Guide

Let AI analyze your survey responses and suggest a coding guide with categories, definitions, and examples. Uses dual-AI methodology for academic rigor.

How It Works

Auto-suggest uses qualcode.ai's dual-rater methodology to analyze your survey responses and identify common themes:

1. You provide your data: Upload a CSV or Excel file with open-ended survey responses
2. Two AIs analyze independently: OpenAI GPT-5.2 and Anthropic Claude Opus 4.5 each analyze a random sample of your responses and suggest categories
3. Semantic merge: A third AI pass performs semantic reconciliation—matching categories by meaning, not just name. Categories identified by both AIs are flagged as high-confidence; categories from only one AI are flagged as low-confidence
4. You review and refine: Rename, merge, split, or delete categories. Add or remove examples. You control the final result
5. Create your guide: Apply the suggestions to create a new coding guide, ready for coding runs

Why Two AIs?

Using two independent AI models mirrors the inter-rater reliability approach used in traditional qualitative research. When two human coders independently analyze data and agree on themes, those themes are more likely to be meaningful and valid.

The same principle applies here:

• Categories both AIs identified are marked "High" confidence - they represent clear, robust themes in your data
• Categories only one AI found are marked "Low" confidence - they may represent valid but less obvious themes, or false positives

Getting Started

To use auto-suggest, you need a data file with survey responses already uploaded to your project.

1. Navigate to your project and open your data file
2. Click Suggest Coding Guide (sparkle icon) in the data file header
3. Select the column containing your open-ended responses
4. Choose your analysis mode (Quick or Thorough)
5. Adjust the sample size if needed
6. Click Generate Suggestions to start the analysis

Premium AI Models

Auto-suggest always uses our most capable AI models to ensure high-quality category suggestions:

• OpenAI GPT-5.2 with extended reasoning capabilities
• Anthropic Claude Opus 4.5 with extended thinking

These premium models provide deeper analysis and more nuanced category identification than the standard models used for coding runs.

Choosing an Analysis Mode

You can choose between two analysis modes:

Sample Size

You can adjust how many responses are analyzed (10-1000). The default of 300 works well for most datasets. Larger samples capture more themes but cost more credits.

Responses are randomly selected from your data file. This ensures the sample is representative of the full dataset rather than biased toward responses at the beginning or end of the file.

Understanding Results

After analysis completes, you will see a list of suggested categories. Each category includes:

• Category name: A short, descriptive label for the theme
• Description: A definition explaining what belongs in this category
• Example responses: Sample responses from your data that fit this category
• Confidence badge: Whether both AIs agreed (High) or only one suggested it (Low)
• Source: Which AI(s) identified this category

Confidence Levels

The confidence level indicates how robustly a category was identified:

Provenance Badges

Each category shows where it came from:

• Both: Category identified by both AIs (shown with a dual-user icon)
• OpenAI only: Category identified only by the OpenAI model
• Anthropic only: Category identified only by the Anthropic model

Editing Suggestions

The suggested categories are a starting point. You should review and refine them before creating your coding guide.

Renaming Categories

Click on a category name to edit it. You can also modify the description to better match your research questions.

Merging Categories

If two suggested categories are similar or overlapping, you can merge them into one. This is common when the two AIs identified the same theme with slightly different names.

Deleting Categories

Remove categories that do not fit your research needs. You can delete:

• Overly broad categories that would catch too many responses
• Categories outside the scope of your research question
• Low-confidence categories you do not find useful

Adjusting Examples

Each category comes with example responses from your data. You can edit or remove examples that do not accurately represent the category. These examples become training data when you create the guide.

Creating the Coding Guide

Once you are satisfied with your categories:

1. Review all categories one final time
2. Enter a name for your new coding guide
3. Choose whether to enable multi-label coding (if responses can belong to multiple categories)
4. Click Create Coding Guide

Your new guide will be created with:

• All categories you selected
• Descriptions for each category
• Training examples from the AI-identified responses

You can then use this guide immediately for coding runs, or continue editing it in the Coding Guides section.

Best Practices

Before Running Auto-Suggest

• Use representative data: The AI analyzes a sample of your responses. Make sure your data file represents the full range of responses you expect.
• Larger samples are better: More responses give the AI more patterns to identify. At least 50-100 responses is recommended.
• Consider your research question: Have a clear idea of what you are looking for - it helps you evaluate whether the suggestions are useful.

Reviewing Suggestions

• Trust high-confidence categories: When both AIs agree, the theme is likely real and meaningful.
• Scrutinize low-confidence categories: These may be valid but need human judgment to confirm.
• Look for missing themes: AI suggestions are a starting point, not exhaustive. Add categories manually if important themes are missing.
• Combine similar categories: The two AIs might use different names for the same concept - merge them.

After Creating the Guide

• Run a test coding: Try the guide on a small subset of your data to see if categories work as expected.
• Add training examples: If certain categories have low accuracy, add more training examples from your reconciled results.
• Iterate: Coding guides improve over time as you add training data and refine categories.

Academic Validity

A common concern: Is AI-generated categorization academically valid? Here is why qualcode.ai's approach meets academic standards:

The Dual-Rater Principle

Inter-rater reliability is a cornerstone of qualitative research. When multiple coders independently analyze the same data and agree, this provides evidence that the coding scheme is reliable and not just one person's interpretation.

qualcode.ai applies this same principle using two genuinely independent AI systems:

• OpenAI's models (trained by OpenAI)
• Anthropic's models (trained by Anthropic)

These systems have different architectures, training data, and design philosophies. When they independently identify the same themes, this provides meaningful validation - similar to two human coders agreeing.

Human Review Required

Auto-suggest is a starting point, not a final answer. The workflow requires human review:

• Researchers decide which suggested categories to keep, modify, or delete
• Category names and descriptions are edited by humans
• The final coding guide is a human-curated product, informed by AI suggestions

Transparent Provenance

Unlike black-box AI tools, qualcode.ai shows you exactly where each suggestion came from:

• Which AI(s) identified each category
• Confidence levels based on inter-AI agreement
• Example responses that support each category

This transparency allows researchers to make informed decisions and report their methodology accurately.

Suggested Methods Section Text

When using auto-suggest, you might describe your methodology as:

"Coding categories were developed using an AI-assisted inductive approach via qualcode.ai's auto-suggest feature. Two independent large language models (OpenAI GPT-5.2 and Anthropic Claude Opus 4.5) analyzed a random sample of [N] responses to identify emergent themes. A semantic merge step reconciled categories by meaning across both models. Categories identified by both models were flagged as high-confidence; categories identified by only one model were flagged as low-confidence. All suggested categories were reviewed and refined by the research team before finalizing the coding guide."

Frequently Asked Questions

How many responses does auto-suggest analyze?

Auto-suggest analyzes a random sample of your responses (configurable from 10 to 1,000, with a default of 300). The sample is randomly selected to ensure it represents the full range of themes in your data. Larger samples may identify more themes but cost more credits.

What if important categories are missing?

AI suggestions are a starting point, not a complete solution. If you know certain themes should be present, add them manually after reviewing the auto-suggest results. The guide creation interface allows you to add categories at any time.

Can I run auto-suggest multiple times?

Yes. If you want to try a different analysis mode (Quick vs Thorough) or if you have updated your data, you can run auto-suggest again. Each run produces fresh suggestions that you can review independently.

Does auto-suggest cost credits?

Yes, auto-suggest uses credits because it runs premium AI analysis on your data. The cost depends on the analysis mode (Quick: 1x, Thorough: 3x) and the sample size. The base cost is 15 credits for 300 responses. The exact cost is shown before you confirm.

What if the two AIs disagree on everything?

Low agreement between AIs can indicate that your data has diverse or complex themes. In this case:

• Review the low-confidence suggestions - many may still be valid
• Consider running with Thorough mode for deeper analysis and better nuance detection
• Use the suggestions as inspiration and create your own categories manually

Related Documentation

• Citing qualcode.ai - Methods section templates for auto-suggested categories
• Dual-Rater Methodology - How two AI raters improve reliability
• Coding Guide Best Practices - Design effective categories
• Training Versions - Improve accuracy with examples
• Agreement Metrics - Understanding Kappa and Alpha