AI Scoring Criteria Best Practices – Gem

One of the most powerful features of Gem’s AI Sourcing and AI-powered App Review solutions is the ability to refine your search using free-form qualifications. This allows you to evaluate nuanced requirements that would otherwise need manual review from a human, since they can’t be assessed with traditional search methods like filters.

As with many AI solutions, the quality of your input (written qualification) strongly determines the quality of the output (e.g. the profiles that AI Sourcing recommends, to the AI match scores in AI-powered App Review). Learning to write the best possible qualifications will take some time and iteration — but this guide provides some best practices to get you started.

To learn more about how our AI match scores work, check out this article: AI Match Scores.

Sample qualifications

We’ve seen qualifications perform well that are related to:

Skills / Tools
Work / Education history
Interests

For example:

Qualification Type	Examples
Skills / Tools	“Experience using React and Python on the job (not just school projects)” “Has worked with BI tools like Tableau” “Experience with data analysis and manipulation tools like Excel, Google Sheets, SQL”
Work / Education history	“Has previously been a founder or started a business” “At least 3 years of experience as a Software Engineer at a tech company” “Experience in legal roles working with securities and finance” “Experience on people team or in human resources at a tech company” “Degree from a school known for its strong engineering program, like Stanford, MIT, or Caltech”
Interests	“Experience or interest in ___” “Has published research, analyses, articles, or blogs on ___” “Active community member in ___ ecosystem or a current open source code contributor”

Avoid qualifications that can’t be easily determined by reading a LinkedIn-type profile or resume. For example, avoid qualifications that are:

Qualification Type to AVOID	Examples
Very subjective	“went to a good school” “exercises good judgement” “highly adaptable”
About soft skills	“a self-starter who can work independently” “strong interpersonal and communication skills” “creative problem solver”
Not likely to be present on a resume / better evaluated in an interview:	"can communicate complex ideas succinctly to decision makers" “can write idiomatic python code” “track record of working with cross-functional stakeholders”

AI tools in Gem are meant to bring you qualified talent, but it can’t replace your interview process. Many qualifications can only be evaluated for in an interview.

Also, do not include qualifications that directly contradict any hard filters you’ve selected - this will give poor results because the qualification will never be met.

Producing qualifications

General framing: Imagine you’re calibrating a new junior sourcer on this role. What would you tell them to look for in a profile?

Gem AI does best when given the same sort of qualifications you’d give to a human sourcer, so this framing tends to produce great qualifications. You can ask yourself:

How would you describe your ideal match?
Are there examples of what set apart the best talent in your mind?
What specifically should the sourcer be looking for on a resume?

If you need a starting point, use a job posting, an internal role doc, or intake form — but be sure to edit any job requirements you borrow.

The role responsibilities, ideal profile attributes, and must-have qualifications from these resources can be a good starting point to get a list of qualifications. You can copy/paste a bulleted list of these directly into Gem’s intake form, and edit from there. But be careful - many “soft skill” requirements on job postings make for poor AI qualifications.

Think about what you actually look for when reviewing profiles — what sets the best prospects apart from the rest? What’s on their resume? Instruct Gem AI to look for these things.

While the job description is a great place to start, it may not list all the qualifications you’re looking for in an ideal prospect profile. It also may not frame the job requirements in terms of what to look for on a resume alone. Think about what specifically you look for in a profile or resume — whether it’s types of experience, specific skills or tools, or patterns of work history — and write a qualification telling Gem AI how to do the same.

Be specific - for example, instead of saying:

“Has strong knowledge of machine learning"

Tell Gem AI how to figure this out this using only a prospect’s profile:

“Has an advanced degree in machine learning, or extensive on-the-job experience working with machine learning algorithms; ideally has also published ML papers or given talks at ML conferences.”

Formatting qualifications

Write the qualification like you’d write a job requirement on a job posting. Provide necessary context, but keep it concise.

For example, if we were sourcing for a PM, this might be a good qualification:

Good qualification: “3-5 years of experience as a Product Manager at a tech company or startup”

And here are some variations of the same qualification that should be avoided and why:

Too short: “PM experience”
No context, will confuse the AI: “product manager”
Too wordy: “Relevant work experience: we need the prospect to have between 3 and 5 years of experience working as a product manager, as this is a mid to senior role that will be both customer and leadership facing.”
Not formatted like a job requirement: “ONLY find me candidates that have 3-5 years of experience as a PM, and REJECT anyone who is a product marketer”

It can be tempting to write the qualification as if you’re talking to the AI (e.g. “please give me people who ___ and avoid ___”) but this will actually confuse it. Your qualifications are dropped into a template that already has instructions, and

For repetitive or similar qualifications, you may want these scored together so they aren’t over-weighted in the final score. Consider grouping these into a single qualification. For example:

Instead of

“proficiency with Python”
“proficiency with R”
“proficiency with C++”

“proficiency with Python, R, and C++”

Or, if proficiency in only one of these languages is needed:

“proficiency with one of Python, R, or C++”

Looking for more articles on Gem AI Sourcing? Please take a look at AI Sourcing!

Have any issues or questions on this topic? Please feel free to contact your dedicated Gem Customer Success Manager directly or our Support team at support@gem.com.