This ecommerce giant made its billions by automating shopping service, but when it came to recruiting it made three crucial mistakes that lead to bias.

According to sources close to the project, it was obvious from the first year that AMZN.O – Amazon’s Recruiting AI – did not like women… like, at all!

The classified project quietly started in 2014, the Seattle company sought to create in-house computer programs to review and score candidates, sources told Reuters. “Everyone wanted this Holy Grail,” one source shared. “They literally wanted it to be an engine where I’m going to give you 100 resumes, it will spit out the top five, and we’ll hire those.”

In much the same way that customers rate products, AMZN.O rated candidates from one to five stars. However, a year into the experiment a gender bias became apparent, especially for software developers and technical posts.

The problem was in the data – the algorithm was feeding on a decade of almost all male resumes and concluding that the ideal candidate is a man, or rather, not a woman. AMZN.O would dock points from graduates of all women’s colleges and downgrade resumes with the word woman/women like “women’s chess club captain.”

Sources say that the algorithm was edited to be neutral to these specific terms, but there was still the fear that the program would teach itself new ways to detect femme resumes and continue to grade them lower.

And though Amazon owes most of its success to its ability to automate everything from warehouse management to pricing, the project was scrapped last year as executives lost hope that AMZN.O could ever be functional. The sources, who only agreed to speak with Reuters a year after the project ended — and under complete anonymity — maintain that no hiring decisions were made using the bias AI.

For some, this story is proof that we aren’t ready for AI in recruiting, and indeed there is still much to learn. Computer scientists like Nihar Shah, who teaches machine learning at Carnegie Mellon University, warn that an algorithm is easier to make than to control.

“How to ensure that the algorithm is fair, how to make sure the algorithm is really interpretable and explainable – that’s still quite far off.”

However, according to 2017 CareerBuilder survey, 55 percent of US HR managers said that AI will be a regular part of their work within the next five years. So is the solution really to avoid artificial intelligence in recruiting all together, or are there some lessons we can glean from this AI debacle? For further insight, we talked to the director of product for SmartRecruiters, Hessam Lavi.

“Developers of this type of systems have an enormous responsibility to prevent negative biases to shape the artificial intelligence they want to produce,” says Lavi. “So, proper training needs to take place to learn not just the technical and process effects of artificial intelligence, but how AI will affect natural beings as well.”

Lavi, who recently headed the team in building SmartAssistant, the first recruiting AI native to an ATS, sees three crucial mistakes when it comes to AMZN.O

1. Thinking the bias is coming from the machine: Negative biases are unfortunately part of the recruiting trade, whether from humans or machines – only it’s much harder to detect in people. So, having a system that makes biases apparent is valuable in itself. The AI learns from the data you feed it, so it’s not the program that’s biased so much as the people who made the decisions that the computer is now analyzing. Eliminating the program is not tantamount to eliminating bias.
2. Limiting the data set: The dataset from one company, even one as big as Amazon, just isn’t enough. A singular company may be using bias paradigms unintentionally. The bottom line is, the more data the better.
3. Deriving future predictions from past events: Past-predicts-future AI can work great for domains such as medical imaging that have a very narrow focus, for example, forecasting the growth of a tumor where an AI can be trained to make clear-cut decisions and act as an expert. However, in hiring which involves a wide range of factors, this type of assumptive AI tends to emphasize biases of the past. If you only had men hired in the past, the algorithm may assume it’s because they are the best people for the job and will continue prioritizing them for future positions.

His best advice? Avoid the black box!

“When we built SmartAssistant we split up the decision processing into smaller, distinct components,” says Lavi. “For example, one component would analyze candidates’ industry experience, one would examine education, one would evaluate soft skills, and so on. Through creating these stand-alone units, we can trace negative outcomes back to their origin and understand why they are happening.”

“We believe the final decision in the recruiting process will be made by humans for the foreseeable future,” Lavi affirms. “But, AI has the ability to automate many of the repetitive tasks and winnow down the stacks of resumes that overwhelm recruiters and cause them to lean on their negative biases. AI technology is much more than just automating tasks and it can teach us about how we make decisions and point out shortcomings in our abilities.”

For this independent music distributor, staying relevant in a continually disrupted industry means taking a non-traditional approach to client needs and staffing.

In the late 90s and early 2000s, physical formats like CDs dominated the music industry, comprising over 99 percent of all worldwide revenue. The Big Six labels—Warner, EMI, Sony, BMG, Universal, and PolyGram—controlled nearly all music distribution rights and reaped all profits. By the mid-2000s, however, it was a different world. iPods replaced Discmans, iTunes replaced record stores, and digital services began to take hold as the new industry standard.

When Napster blew up the Big Six oligopoly in 2001, it took the bottom out of record-store revenues and opened the market for the kind of on-demand streaming we enjoy today. While the new consumer norm is to create networks on streaming platforms like Apple Music, Spotify, or YouTube, industry profits have not recovered. Despite three years of consecutive growth, music industry revenues in 2017 were just 68 percent of what they were in 1999. As a whole, the industry raked in $17 billion last year, up 8 percent from the previous year, but those figures fall short compared to the $25 billion of 1999.

This mix-and-match, playlist-based behavior reflects the free-flowing nature of today’s digital economy – complete with the struggle to profit from subscription models and subsist on paltry ad sales. So how can an independent, digital music distributor navigate such an inhospitable industry and still turn a profit?

In a world where even the definition of “record label” is up for debate, Ingrooves Music Group walks the line between distribution company and marketing agency, liaising between artists/labels and third-party retailers like Spotify or Apple Music. The company also tracks the analytics, managing rights and royalties for its clients.

During the early years of conflict between analog and digital formats, Ingrooves Music Group was founded when entrepreneur Robb McDaniels saw a business opportunity in the nascent digital music scene of 2002: Music was quickly becoming accessible to anyone with an internet connection, and new bands, DJs, and other artists needed help promoting and distributing their music across the web. As part of his mission to create the “digital record label of the future”, McDaniels sought to bridge the gap between music artists and music retailers, offering a platform for more accessible music distribution and artist representation: An artist or label sends Ingrooves a song or album, which is then converted into the proper format and pushed out to iTunes, Spotify, Amazon, and 600 other digital retailers. When someone buys or streams the music, the artist’s royalties are passed along to ­Ingrooves, which pools them and then reimburses the artist. Ingrooves profits by taking a percentage (10%–30%) of an album’s wholesale cost (A typical album ­on iTunes retails for $10 and wholesales for $7).

In the midst of this race for digital domination, people remain a key element of the music industry, and Ingrooves understands that at the end of the day, it’s not algorithms, but people, that truly matter. “That’s just the core of HR, and our job is to take care of our clients,” says David Kroes, who had been heading Ingrooves’ hiring drives for technical and creative teams since January, 2016, captaining recruiters, hiring managers, and employees through the modern music industry.

With offices in Los Angeles, New York, London, Berlin, Oslo, and Victoria, BC, Ingrooves currently generates around $150 million in annual revenue and holds a two percent market share of recorded music in the US. To stay on top of competition within the industry, Ingrooves is pressured to manage a highly successful global team. A poor hire could cost the company, and in an industry feverishly scraping away at profits, that is one mistake Ingrooves cannot afford.

Hundreds of candidates flood Kroes’ inbox on a regular basis looking to break into the music scene. Unfortunately, the reality of pacing such an uncertain industry is that many employees become obsolete and burn out much faster than in other organizations. “The music industry is a lot like surfing” says Kroes. “People keep coming back even though it’s not the most forgiving environment, but their love for the industry is what keeps people loyal.”

The Ingrooves business model is a far cry from that of the brick-and-mortar retailers like Tower Records—the last of which shuttered in December 2006, more or less the final breath of the American record industry as we knew it—but as Ingrooves emerged, its practices were already forward-thinking, earning mainstream recognition as one of the first independent distribution companies to partner with iTunes. The music industry, however, is a fickle beast. In a few short years, on-demand streaming would incite another format revolution that, to this day, shows no signs of slowing.

Figures released by IFPI’s Global Music Trends 2018 report show that streaming accounts for 38 percent of total recorded music revenue, making it the industry’s single largest profit driver for the first time ever, with 176 million users of paid streaming services contributing to a 41 percent growth over last year.

With real money to be made from this new format ($5.6 billion in 2017) companies like Spotify, Apple, Google, and Amazon all currently offer dedicated, paid streaming platforms. Additionally, video streaming services like YouTube comprise more than half of on-demand stream time, attracting 1.3 billion total users in 2017, according to the IFPI report. How does Kroes plan on keeping Ingrooves ahead of upcoming changes within the industry while supporting the workforce powering a global music distribution and marketing company?

“At Ingrooves, we went through a situation where we were a teenager for a long time,” he says. “We were a startup and we never really outgrew the startup mode. After I joined the team we really started to realize our opportunity to become an adult.”

Pushing the company into the next phase wasn’t about changing out the old guard for fresh blood, it was about building a “culture of adults” that are adaptable and able to think on their feet. “The culture at Ingrooves is still evolving, and new talent is bringing it in,” says Kroes. “You always want to bring in new perspectives and balance that with experience. To me, experience is the best teacher.”

Kroes came onboard in the midst of company-wide restructuring that required a challenging personnel shift and revamp of its company brand. With a strong grasp on the analytics and reporting side of licensing, publishing, and distribution, Ingrooves doubled down on its high-impact marketing, PR, and promotion, working closer with artists and other industry organizations, to “help them understand and anticipate their needs” before they even knew what to ask for, a paradigm that Kroes carried over into his recruiting practices. Kroes leveraged his decade-plus of experience working in TA, as well as a strong background in philosophy, to reshape the organization’s approach to hiring.

“Media companies, whether TV, movies, or music, are very much people businesses,” he notes. “A lot of talent is not cookie-cutter.” In order to hire both technical and creative roles in tandem, Kroes needed a non-traditional approach, which, he says was made possible with tools like the SmartRecruiters Talent Acquisition Suite. “A good recruiter should be looking for ways to find talent that maybe other people haven’t, and SmartRecruiters allows you to experiment with a smart approach that pushes the limits of what the tool is designed to do,” says Kroes. “It gives you more insight into the state of things.”

While predicting what’s next to come in the music industry is far from science, Kroes is confident that Ingrooves Music Group will continue to adapt as the industry embraces on-demand streaming. Issues such as diminishing profit margins will be a challenge, but Ingrooves is ready to face the challenges like the now grown-up company that it is, one which, with Kroes at the helm, can finally have its HR Head say that the company, firmly, “knows the path forward.”

Fall means beautiful heaps of russet leaves – ripe for the jumping-in but even more exciting, Fall means more features in the SmartRecruiters Ecosystem. It’s here! The latest Product Release! Integrations that will save time, encourage collaboration, and boost candidate experience.

When we say integration we mean partnership with the platforms you already use like Facebook, Google Job Search, and Slack so you can work where you work best with the full power of SmartRecruiters’ Talent Acquisition Suite behind you.

Open up a channel in Slack with other interviewers, use Facebook to create a job marketing campaign, or narrow down your job hunt by commute time with Google Job Search – all these things are now possible within the SmartRecruiters ecosystem

Our OS ecosystem will now include these major players to enhance each step of the recruiting process from attracting to selecting to hiring. It’s all about saving you time so that you can invest in relationships with the right candidates. Because we think technology should help us be better more personal humans.

And that’s why these small tweaks and add-ons are actually a big deal!

Maybe you’re waiting for a promising candidate to graduate you can use the Defer Candidate feature to resurface them at the right time and on the other side if you know a candidate is a poor fit immediately (heartbreaking but it happens) you can utilize the Delayed Rejection option to observe an appropriate period of consideration. Recruiters can also expedite the vetting process with Screening Automation that uses enhanced screening questions to sort through applicants more efficiently.

We give you the reigns with the right amount of flexibility and control to tailor the experience to the needs of your business. In the end we want you to feel in command of the hiring process. It should be a good for everyone.

One key component of how we fit with your business is the language we use. Now pretty much wherever you are you can find the front and back end languages you need from our system. We are adding more languages to the platform’s back office functionality, including traditional Chinese, Hungarian, Japanese, Portuguese and Slovak, on top of the 12 other languages it currently supports, and is adding new candidate languages, including Greek, Indonesian and Ukrainian, to the 30 other candidate languages currently supported.
Feature Close-ups

Facebook – As our collaboration with Facebook develops we continue to support customers with the out-of-the-box Facebook app, full Share to functionalities and most recently the ability to create targeted advertising campaigns with the power of facebook and the support of SmartRecruiters marketing team.

Google Jobs Search –  Take your jobs to the top of the search list! Our Alpha partnership with Google makes it easier for our custom jobs to be discovered by candidates. Part of this partnership includes improved access to detailed information like job location and commute time via Google Maps, with all our job ads. With the new Google Job Search functionality, candidates won’t have to navigate cumbersome job boards to find relevant opportunities which means more completed applications.

Slack –Reduce clicks and encourage collaboration. You can now automatically create job channels in Slack from the SmartRecruiters platform based on hiring team. Participants are also able to unfurl links via the Slack app, providing direct access to candidate and job profiles for recruiters, hiring managers and interviewers. No switching in and out of apps!

Defer Candidate – Keep your company’s candidate pipeline relevant. Resurface candidates at the appropriate time within the SmartRecruiters platform. No externally-triggered reminder needed!

Delayed Rejection – This upgrade to the Rejections feature enables recruiters to delay a rejection email to a candidate, so that they still have the feeling of being considered instead of simply “screened by a robot”. This is a way for recruiters to respect the time investment of the candidate and ensure a positive experience even when delivering bad news.

Amazon Alexa – SmartRecruiters is excited to integrate Alexa, Amazon’s virtual assistant into our candidate platform to make it easier for candidates who use Echo to check their application status and send messages to recruiters using Alexa voice commands. This feature will be accessible to everyone at the start of next year is currently on available to select users.

Wherever communication is happening is where we want to be and we look forward to integrating with more and more platforms in the future. You choose where you do the work and we help with the how. Our OS is built to expand as the needs of our customers change and grow with a dynamic ecosystem that’s built for innovation and a team that’s always looking ahead. Winter is coming but unlike the gorgeous autumn leaves, these features are here to stay and will only get better.

Link to full press release!