Valence and danceability: the strengths and shortcomings of Spotify’s algorithms

 Radiohead performing at the Coachella Valley Music and Arts Festival in Indio, California on May 1, 2004. The last show of their Hail to the Thief tour. (g_leon_h/CC 2.0 License)  Radiohead performing at the Coachella Valley Music and Arts Festival in Indio, California on May 1, 2004. The last show of their Hail to the Thief tour. (g_leon_h/CC 2.0 License)

*this is meant to be a quick comparison, not an in-depth critique

Though 99 percent of users have no reason to access Spotify’s developer tools and data-driven Insights, they offer a deep dive into the characteristics of individual and societal music tastes and trends. Spotify Insights allow you to measure the ‘danceability’ and ‘valence’, or positivity, of a song. Though the tools are powerful, comparisons of covers and alternate takes reveal the strength of the ‘valence’ measure, and the comical, albeit inconsistent, failings of the ambiguous concept of ‘danceability’

The tools on Spotify Insights come by way of a subsidiary, The Echo Nest. Spotify Inc. purchased The Echo Nest, a self-described music intelligence and data-performance company in 2014. Since then, the streaming juggernaut has incorporated the company’s music tools into their Spotify Insights page.

Users can analyze playlists and songs for standard metrics of BPM or release date; but, more interestingly, the ‘danceability’ and ‘valence’ of songs illustrates something I would’ve thought difficult—the emotionality and, for the lack of a better word, vibe of a song.

Valence v. Danceability

A 2013 post from The Echo Nest’s Eliot Van Buskirk, now migrated to the Spotify Insights page, gave a simple explanation of ‘valence’ as it relates to the data. For The Echo Nest, valence is “to describe whether something is likely to make someone feel happy (positive valence) or sad (negative valence).” He went on to admit “It’s no easy feat to have a computer listen to a song in three seconds and determine its emotional valence, but we’ve figured out how to do it” with one notable, human-addition to the process being the use of a “music expert classify some sample songs by valence, then use machine-learning to extend those rules to all of the rest of the music in the world, fine tuning as we go.”

Buskirk’s data shows the relative ‘valence’ of popular music has remained near-neutral since the 1950s.

I came to this concept through a video from online music-critic, The Needle Drop, discussing a 2017 blog post from ‘RCharlie’ discussing Radiohead—a band known for producing some of the most depressing records to date.

‘RCharlie’ found the use of valence-alone to be of limited reliability, and adapted the measure to account for lyrics, i.e. identifying words and lyrics typically associated with “sadness.” The results were ostensibly pretty accurate. Radiohead’s latest album, 2016’s “A Moon Shaped Pool” came out as far and away the saddest album—a fact one listen will confirm.

Buskirk’s data shows the relative ‘valence’ of popular music has remained near-neutral since the 1950s.

‘Danceability’ would seem, but is not necessarily opposed to ‘valence’. A favorite band of mine, LCD Soundsystem produces songs crafted for an analog sense of ‘danceability’, but with lyrics often rivaling Thom Yorke & Co. for depression.

A 2010 blog post from members of The Echo Nest team describes ‘danceability’ as inherently subjective: “The Echo Nest defines danceability as the ease with which a person could dance to a song, over the course of the whole song. We use a mix of features to compute danceability, including beat strength, tempo stability, overall tempo, and more.”

As someone who invests more attention to the lyrics of songs, I’m curious how covers stack up to originals, and how alternate takes from the same artists change things. Really, how much weight are lyrics in how danceable or sad a song is—or how we perceive those metrics. Covers often slow down and rework songs, revealing the power (or lack of power) of lyrics. In no case is that more clear than in downtrodden songs.

Covers

One of the best known covers is Jeff Buckley’s version of Leonard Cohen’s “Hallelujah”. If you’re attempting to dance to either, you’re probably a good candidate fro psychiatric treatment. Cohen’s words are depressing, recounting Samson and Delilah. Yet despite the use of Cohen’s lyrics, Buckley’s version is substantially lower in valence, i.e. has a negative valence (i.e. is more sad, according to the Spotify/Echo Nest algorithm). Cohen’s version relies on his well-known Greek choir backing, and has a bit more variation musically. That might account for the difference. Both are well below a song of average-valence, or neutrality in regards to positivity/negativity (at 50).

Screen Shot 2018-05-29 at 10.15.30 PM

A more telling example would be a song whose cover is radically different, and more importantly, with the inclusion of a song irrefutably belonging to the ‘dance’ category. Robyn’s “Dancing on My Own” provides that. Whereas Robyn’s lyrics are somewhat ambivalent, certainly edging toward sad, her music is upbeat and crafted for danceability. Callum Scott’s cover, however, turns the song into a sappy piano-ballad.

Surprisingly, despite being polar-opposites in regards to non-lyrical music, both versions are nearly identical. Robyn’s original scores a danceability of 68 versus Scott’s 69, whereas the original scores a 26 in valence, with Scott’s at 20.

This similarity shows the limitation of this algorithm. Despite being very good at measuring valence, as both songs are nearly equal in that regard, the danceability measure is bizarre. Placing Robyn’s version as one-notch below Scott’s in this metric is comical. One listen defeats that argument.

Alternate Takes

The song referenced earlier, “True Love Waits” by Radiohead is the epitome of a low-valence record. It is depressing, in both music and lyrics. Them Yorke premiered the song in the 1990s, but waited until 2016 to release the track on a proper album. In between, he transitioned it from an acoustic guitar ballad to a phantasmagoric piano ballad. The first recording has an air of potential change, while the second is resigned and defeated, both containing one of Yorke’s finest and saddest lyrics, “I’m not living, I’m just killing time.” The second record came after Yorke and his partner and co-parent of 23 years, Rachel Owen separated, and a half a year or so before her untimely death from cancer. That lamentation is painfully-clear in the reworking of this song.

All of this is to say, this record went from typical Radiohead-sad, to a remarkable new level.

The inclusion of danceability here is just for consistency. Both versions make “Hallelujah” sound like “Beat It”. The 2003 live-recording of the song has a danceability of 37 versus 28 for the later version, and a valence of six compared to four for the 2016 version.

The similarity in valence, again, shows the strength of this metric. ‘Sad’ music can have such a wide-variety of sounds, from ABBA to Otis Redding to Nas to Amy Winehouse to Mozart to Beyoncé. The Spotify metric purposefully does not rely on key alone, as a minor key gives an almost instant impression of sadness, from which the lyrics might deviate.

As for the variation in ‘danceability,’ I put that more to the difficulty in assessing this quality, which stands to be far more subjective than even valence. In my Spotify Top Songs of 2017, the most danceable songs was “Oh Devil” by Electric Guest at 90, the least was “Lust for Life” by Iggy Pop at 20. For valence, the most positive was “Scarlet Begonias” by the Grateful Dead at 92, the most negative being “Waves” by Kanye West at 6. Just from a music listener point of view, I again find the valence numbers to be far more reflective of reality. It’s hard to get more positive than “Scarlet Begonias“.

*The Spotify Insights page can be found here.

“A Change is Gonna Come” as Gospel: Analyzing Sam Cooke’s Opus

A piece I wrote several years ago for a term paper during a course on African-American history, edited for online publication.


Within the first few bars of “A Change is Gonna Come” a yearning and lamenting vocal rises above the orchestra. The song’s origins are often told mythically, Cooke responding to an incident of potent racism, recording the song with spontaneous conviction as a sudden and stark contrast to his popular, romantic songs.

Cooke’s move toward more personal, observational songs was not sudden, as evidenced by the inclusion of social themes in songs such as “Chain Gang” in 1960, four years before the release of his opus. Following Cooke’s premature death in 1964, this seminal work has been heralded as a cornerstone of the protest song genre. While the song has come to function as a powerful protest song in the Civil Rights Movement and within other activist causes, there is little evidence to suggest that Cooke intended the song to fit the mold of the traditional protest songs of the era.

Though Cooke spoke infrequently on the song, his roots in gospel elucidate a clear counterfactual to the oft-referenced consensus. Cooke’s ostensible reluctance to sing the song live, use of  personal narration, and most importantly, his upbringing and fruitful early career in gospel and spiritual music more accurately position “A Change is Gonna Come” as a contemporary addition to the spiritual and gospel canon.

Cooke’s lamentations, mixed with an uncertain hope, sound far more similar to the music of his early career than protest songs of the 1960s. He calls up to the divine in the song, looking to society and the law for a hand in the fight for civil rights and against white supremacy. Though he drew inspiration from popular protest songs of the 1960s—most notably Bob Dylan’s “Blowin’ in the Wind”—the song reflects Cooke’s own observations and yearnings, as seen through the lens of his particular life and upbringing.

Sam Cooke was born in Clarksdale, Mississippi on January 22, 1931 to a reverend father, Charles Cooke (Sam Cooke added the ‘e’ later in life), and his mother, Annie Mae Cook.[1] Unlike many musicians, Cooke’s religious and musical upbringing were one, with his soul steeped in the sorrow, emotion and hope of spiritual and gospel music.

Cooke’s parents, according to his biographer, Peter Guralnick, met at a “convention at which [Rev. Charles Cook] was preaching, and they started going to church together.”[2] Cooke’s parents met in the church, lived in the church and raised their children within the church.

His exposure to gospel and spiritual music came at a transitory point, in which traditional songs were becoming infused with more contemporary sounds, as groups’ “driving attack mimicked the sound, as well as the message, of gospel preaching” while the “repertoire … frequently sprang from more accessible personal testimony.”[3] During his youth, gospel and spirituals began to take on a more accessible sound, with Cooke leading one of the more notable groups of this era, the Soul Stirrers, early in his career.[4]

When Cooke transitioned to pop music during the late 1950s, his music’s orientation tilted away from the religious, and toward a romantic tone. This tone, closer to that used by Frank Sinatra and other popular musicians, proved more appealing to white audiences, and thus more commercially viable.

As Cooke would later tell Dick Clark in 1964 on American Bandstand, he left the Soul Stirrers and spiritual genre for the economic potential of popular song.[5] The economic incentive involved with switching over to popular music impeded Cooke’s ability to produce music reflecting the strife of the day.

For Cooke and others, the economic risk involved with penning and releasing “A Change is Gonna Come” was clear, though, in returning to his earlier roots, the power of this influence diminished. When Cooke penned “A Change is Gonna Come” his friend and fellow Soul Stirrer J.W. Alexander remarked:

We might not make as much money off this…but I think this is one of the best things you’ve written.[6]

Though Cooke would have understood the economics involved with releasing “A Change is Gonna Come,” his response to Alexander’s concern reveals his state of mind: “I think my daddy will be proud [of the record].”[7]

When “A Change is Gonna Come,” came to airwaves shortly before Cooke’s violent death in December of 1964, it stood as a strong contrast to the pop-music that had brought him fame and fortune.[8] The record echoed the lamentations and religious testaments of his earlier gospel and spiritual roots, communicating a desperate plea for change in humanity, not the direct confrontation and questioning present in the pure protest music of the day.

A reluctant release, Cooke’s observations and reflections served as a meditation on the Civil Rights movement with his death. This song sits at the intersection of Cooke’s childhood, early career, experiences as a black man in mid-century America, providing conflict with his platform as one of the most notable entertainers of his day.

Contextualizing song as a gospel or spiritual work acknowledges the personal and social history contained in the music and lyrics, remedying the popular belief that the record marked an awakening in his career, as if the popular music icon suddenly found a breath of inspiration. Often this change is explained through a notable event in Cooke’s life, in which a whites-only motel refused him board, resulting in an altercation with the police in October 1963.[9]

The newspaper coverage of the arrest was minimal, with a reporter noting, “Sam Cooke…a Negro band leader; his wife and two associates were arrested for disturbing the peace” while trying to book a room at a “white motel.”[10]

The notion that racism could transcend even his immense fame, wealth and status as a member of the human race, clearly, from his biographers and contemporaries such as Bobby Womack, inspired this record.[11] However, the switch to personal music, reflective of the times he lived in, represented the result of a long journey back to his gospel roots. Such music relies on personal narration, and a plea or reflection of troubles, in the hopes that the Lord will intervene and end or alleviate suffering. The hope for amelioration of one’s situation is clearly reflected in “A Change is Gonna Come.”

The idea of a radical departure rests on a superficial understanding of Cooke’s canon. In 1960, Cooke wrote and released, “Chain Gang.” Though the song maintains the pop-sound of his contemporary records, it contains only a few lines of his typical romantic prose. Instead, Cooke sings:

All day long they work so hard/Till the sun is goin’ down/Working on the highways and byways/And wearing, wearing a frown/You hear them moanin’ their lives away.[12]

Though the accompaniment maintains the semblance of a commercial sound, the lyrics reflect Cooke’s own observations of a chain gang, certainly containing a majority of African-Americans and other minorities, working roadside in the heat of the Deep South.[13]

This was Cooke’s first experiment with the commercial viability of a record reflecting the realities of the time. The record reflects an observation of the criminal justice system, noting its role as a “vehicle of social control” and eliciting political themes, absent in Cooke’s earlier popular music work.[14] In a 1964 television interview with Dick Clark, Cooke, when asked what the secret of his music is, responded:

I think the secret is really observation. Well, if observe what’s going on and try to figure out how people are thinking and determine the times of your day, I think you can always write something that people understand.[15]

Though the song does not have the same clear message as “A Change is Gonna Come” and relies on a romantic tone and lyrical layout, the change from previous songs such as “Bring it on Home” and “Having a Party” is striking. The transition four years later seems to be less of a radical change, and more of the polished end-product of this experimentation with a personal, social form of music.

Cooke’s transition back to music with a gospel underpinning also reflects his ability to blend musical styles with grace. While with the Soul Stirrers, Cooke took traditional gospel and spiritual music and “extend[ed] its appeal to a younger generation of fans and singers.”[16] He used popular music structure and vivaciousness in his renditions of gospel music, choosing to combine musical genre, rather than stick to one genre or another. This distribution of this blending, clearly in favor of gospel early in his career then popular music later, swung back towards the gospel end of the spectrum with “A Change is Gonna Come.”

While with the Soul Stirrers, Cooke sang lead vocals on a stirring rendition of “Nearer to Thee,” in which he sings, “There’s a story in every song we sing/Songs that are known to lift heavy burdens.”[17] This weight of this prophetic observation came to light years later in the 1960s, during which the Civil Rights Movement adopted Cooke’s “A Change is Gonna Come” as an anthem of change, using Cooke’s words to help lift the heavy burden of institutionalized, systemic racism in America.

As Dr. Christopher Trigg argues, “Protest songs are sung from a position inside social change.”[18] Cooke’s lyrics do not argue solely from this position, but instead place the “speaker at a distance from change, outside [the protest].”[19] Cooke sings in the personal pronoun, saying “I was born by the river” and “Then I go to my brother” as opposed to most protest songs of the era, such as those sung by contemporaries, which adopt a third-person narrator and tone.[20]

The use of an external figure, the brother of Cooke’s composition, is also historically tied to spirituals. The message in these songs is supposed to rouse the emotional depths of individuals suffering the same strife as the narrator, while calling attention to the existence of an oppressor, or in Cooke’s case, those in society who could help but do not. Dr. James Stewart said, of political commentary in black popular music, such as Cooke’s, that artists “communicate simultaneously with internal and external audiences” reflecting a “modern variant of the type of double entendre reflected in many…Spirituals…”[21]

While African-American audiences would identify with Cooke’s narration and understand the same strife he sings of, white audiences might, upon reflection, understand the brother to be a condemnation of silent passersby and the more recognized establishment, who do nothing to help and everything to hurt the African-American community.

The message, like that of spirituals, especially those intended to provide “resistance and…guidance,” could be interpreted differently by two different audiences.[22] Cooke’s song provides a message of shared suffering for African-American audiences, while translating as a condemnation to white perpetrators, and worse, silent witnesses.

Protest songs are, by their socio-political nature, meant to be sung to the masses in order to inspire and motivate, beyond the walls of a home or church. When Cooke first played the record for a young Bobby Womack, his protégé remarked that “It feels like death…it feels eerie, like something’s going to happen.”[23] Womack’s meditation on the change in tone and lyrical content of “A Change is Gonna Come” is evident.

To this comment, Cooke responded “that’s why I’m never going to sing that fucker in public.”[24] If Cooke had intended the song to be one fitting in the protest canon, it seems unlikely that he would have intended for it to be sung publicly.

Cooke struggled even with publishing the song, only doing so ten months after recording, but with passionate arguments, ensuring that the entire record be published—with no cuts; this insistence on including all verses, including the final verse which solidified “[the song’s] underlying statement of redemption and belief” proves the depth of Cooke’s meditation on his observations and thoughts, and the lack of urgency on his part to trumpet a song with an imperfect message.[25]

Cooke’s return to his gospel and spiritual roots does not mean that “A Change is Gonna Come” did not become a song of protest. Though the song is now used as a protest song, the frequency of this usage cannot be used to inform the overall understanding of Sam Cooke’s intention in crafting the song, nor can it negate the powerful influence of his own life in the lyrics and melody.

Songs can either enter the protest genre at the intention of their creator, or become songs of protest through popular usage. With Cooke, there can never be a definitive answer at to what Cooke’s intentions were, as he never lived to see his work become a Civil Rights anthem.

In a radio interview with “Magnificent” Montague in 1962, Cooke further foreshadowed, with his often prophetic insight, his later career transition. He said to Montague, in response to a question regarding changes in his career since the days of the Soul Stirrers:

As a singer grows older, his conception goes a little deeper because he lives life and he understands who he’s trying to say a little more.[26]

Montague’s interview with Cooke ended with a request for eight bars of soul.[27] Cooke softly hummed a haunting melody, void of lyrics, but soaked in emotion. The same emotion he evoked in this short, eight-bar clip is heard on “A Change is Gonna Come” as well.

Though the song has become one of the greatest protest songs of the century, it began life as a call back to the early music of Cooke’s life and career, and the lamentations wrought from intense trials and tribulations. While the spirituals and gospel anthems sung in his father’s services were inspired, chiefly, by the days of slavery, Cooke brought the trials of modern African-Americans into the canon, crossing genres in his characteristic style.

“A Change is Gonna Come” stands as an anthem of hope for change in the light of tremendous adversity and struggle. Cooke’s words, when taken out of the context of protest usage common after his death, reflect a plea and an expectation that change would have to come in the Civil Rights movement, lest the African-American community’s brothers in the establishment ignore their plight for another era.

The vocals express disbelief at the extent of the dismissal of oppression, speaking almost as if the entire black experience has been a phantasmagoric horror, with the injuries and strife continuing on for an inordinate amount of time in the face of an establishment capable of outlawing such discrimination and pain.

Sam Cooke’s work stands as a classic in the modern gospel and spiritual canon, reflecting the narrator’s shaken faith in God and the system of justice and freedom. The final threads of belief, wrung out of Cooke’s voice, look upwards and in the eyes of his fellow brothers, searching for the glint of change in those watching eyes.

When he is “knocked back down on his knees” Cooke reflects the continued breakdown of his community, whose transient move forward during Reconstruction fell in the grasping hands of white supremacy and institutionalized racism, borne into the laws to make freedom appear selfsame to slavery.[28] Cooke’s song serves as the end of his transition back to personal and emotional song.

His reliance on observation and personal understanding were key in the creation of his opus, supplying the raw emotion needed to craft a song as powerful as “A Change is Gonna Come.” This song cannot be seen as one forged to communicate and bolster protest; instead, it must be seen as a product of Cooke’s life, as an embittered meditation on the quagmire of progress.


[1]Peter Guralnick, Dream Boogie: The Triumph of Sam Cooke, (eBook ed., New York: Little, Brown and Company, 2008), 24.sssss lyrics speak of a shaken faith in justice and God, and an observation of the troubles and spel or spiritual song. In place

[2] Guralnick, Dream Boogie…, 24.

[3] Ibid., 48.

[4] Ibid., 14.

[5] American Bandstand: Sam Cooke Performance and Interview, Television, Anonymous Web: Dick Clark Productions, 1964.

[6] Guralnick, Dream Boogie…, 630.

[7] Ibid., 630.

[8] Ibid., 1002.

[9] “Negro Band Leader Held in Shreveport,” New York Times (1923-Current File), Oct 9, 1963, 1963, http://search.proquest.com/docview/116550630?accountid=14518.

[10] Ibid.,.

[11] Guralnick, Dream Boogie…, 972.

[12] Sam Cooke, “Chain Gang” (Audio file), from Swing Low, https://www.youtube.com/watch?v=RmZdvVnMXCc (accessed 12/1/15).

[13] Guralnick, Dream Boogie…, 530.

[14] James B. Stewart, “Message in the Music: Political Commentary in Black Popular Music from Rhythm and Blues to Early Hip Hop,” The Journal of African American History 90, no. 3 (2005): 206.

[15] American Bandstand: Sam Cooke Performance and Interview, 1964.

[16] Mark Burford, “Sam Cooke as Pop Album Artist—A Reinvention in Three Songs,” Journal of the American Musicological Society 65, no. 1 (2012): 113-178. doi:10.1525/jams.2012.65.1.113: 121.

[17] The Soul Stirrers, “Nearer to Thee” (Audio file), live recording, https://www.youtube.com/watch?v=aXt-I2bIdZE (accessed 12/1/15).

[18] Christopher Trigg, “A Change Ain’t Gonna Come: Sam Cooke and the Protest Song,” University of Toronto Quarterly: A Canadian Journal of the Humanities 79, no. 3 (2010): 991-1003. doi:10.3138/utq.79.3.991: 994.

[19] Ibid., 994.

[20] Sam Cooke, “A Change is Gonna Come” (Audio file), from Ain’t That Good News, https://www.youtube.com/watch?v=nOYuhLNwh3A (accessed 12/1/15).

[21] Stewart, “Message in the Music…,” 199.

[22] Stewart, “Message in the Music…,” 199.

[23] Guralnick, Dream Boogie…, 906-907.

[24] Ibid., 907.

[25] Ibid., 1002.

[26] Nathaniel ‘Magnificent’ Montague Interview with Sam Cooke, Cooke, Sam and Nathaniel Montague, 1962, Radio KGFJ, (Radio).

[27] Nathaniel ‘Magnificent’ Montague Interview with Sam Cooke, 1962.

[28] Cooke, “A Change is Gonna Come” (Audio file).