Author Topic: Quartz Article - Algorithm-Driven Fiction  (Read 5283 times)

Tom Wood

Quartz Article - Algorithm-Driven Fiction
« on: November 20, 2018, 11:05:50 PM »
The article is a little unfocused, but it does point out how Amazon can use all the customer/reader data that it collects to make predictions about buyer behavior. This isn't about using AI to write books, but rather to use AI to analyze the potential popularity of human-written fiction.

https://qz.com/1453112/amazon-has-everything-it-needs-to-make-massively-popular-algorithm-driven-fiction/

Quote
Amazon’s Kindle e-reader, first released in 2007, is a data-collection device that doubles as reading material. Kindle knows the minutiae of how people read: what they highlight, the fonts they prefer, where in a book they lose interest, what kind of books they finish quickly, and which books gets skimmed rather than read all the way through.

Quote
Like Kindle, Audible is a trove of data. Its app knows when you drop off and stop listening; when you speed up, perhaps because the reading is too slow; when you rewind, maybe because the reading is too fast or the content is too confusing; what passages you bookmark; and what time you put your books on “sleep.”

Quote
In 2013, Amazon bought still another massive dataset: Goodreads. The popular social network for readers today gives Amazon access to 80 million profiles on book preferences, which it could theoretically layer on top of actual reading behavior.

It reminded me of the book The Bestseller Code where the authors describe their method of using computer algorithms to predict popularity:

https://www.goodreads.com/book/show/28220900-the-bestseller-code

The Quartz article includes a link to a discussion about Netflix's use of algorithms:

https://www.gq.com/story/cary-fukunaga-netflix-maniac

Quote
Like Beasts, Maniac will stream on Netflix, which has its own surreal development process. "Because Netflix is a data company, they know exactly how their viewers watch things," Fukunaga says. "So they can look at something you're writing and say, We know based on our data that if you do this, we will lose this many viewers. So it's a different kind of note-giving. It's not like, Let's discuss this and maybe I'm gonna win. The algorithm's argument is gonna win at the end of the day. So the question is do we want to make a creative decision at the risk of losing people."
« Last Edit: November 22, 2018, 12:56:24 AM by Tom Wood »
 

guest957

  • Guest
Re: Quartz Article - Algorithm-Driven Fiction
« Reply #1 on: November 21, 2018, 06:56:53 AM »
Interesting article and subject. Personally, I don't agree with Fukunaga on the algorithms eventually winning out because as extensive as all of this data collected is, there's still a lot out there about human behavior and consumer tastes left undiscovered and un-understood if it's understandable at all (and I don't think it is beyond a certain point).

One could point to publishing's long history, or the history of fiction itself and extrapolate and interpret the data borne of any given story's popularity or even the actual words story consumers have uttered through the ages in awe or disdain of said stories and still one could come away with little in the way of useful information on what's going to be a hit and what isn't. We have centuries worth of that data that's been picked over and re-hashed again and again and still we know only the surface-level aspects of what makes a certain kind of story resonate or what makes 'x' a bestseller. Heck, Hollywood's entire raison d'etre lies in trying to tease out what's going to bring the greatest ROI - get the most bottoms in the most seats at the most multiplexes - and they often wind up frustrated and wrong to the tune of hundreds of millions of dollars thrown away.

I think a lot of the 'formula' stuff, algorithm-extrapolation included, is put forward by those looking to make money off of hopeful storytellers who are looking for that 'easy' button - the shortcut telling them what audiences want which will more quickly vault them atop the bestsellers lists. Meanwhile, the formula peddlers are silently earning multiples of what their eager followers are earning from their fiction. For many writers, there seems to be a better return in selling the promise of being able to calculate and construct what's going to hit big rather than in trying to hit big with their own fiction themselves. If Amazon proves unwise, they could wind up staring into their pool of data long enough that they'll eventually fall into it and drown. Netflix too, for that matter.

I suppose what I'm saying is, the whole thing is squishy, for lack of a better term. People are strange and unpredictable. Tastes vary and can turn on a dime. I've been reading books that I've loved and found fantastically gripping and yet I've put them down and walked away in the middle of particularly engrossing passages. Why? Who knows? Maybe my cold made me drowsy. Maybe a friend arrived. Maybe something distracted me. Maybe something bothered me from 50 pages before and wore at the back of my brain until I felt compelled to stop and think on it. Maybe I smelled and needed a shower right then and there. Maybe my oven was finished preheating. That's just it. There's a million and one reasons why people do this while not doing that. Why they put a book down or pause an audiobook or speed it up or do pretty much anything. Of course, my examples are purely anecdotal, but I doubt they're all that unique. I'm sure I'm not the first to stop consuming something they were enjoying and then forget to go back to it, or they go back to it for any given reason at any given time.

Point is, no one knows why consumers do what they do beyond superficial theorizing. No one knows what's going to hit and what isn't. What's going to grab someone and retain them for 'x' amount of time or for 'x' amount of product. No one can tell what's going to resonate and what's going to fall flat. Entire industries are built on "knowing", but in my opinion it's more like the broken clock analogy. So, Amazon and Netflix can continue compiling data until the cows come home, it won't matter. You won't be able to look at any of it and just conjure a hit from it out of the ether. Sure, you can try, but to me it's a fool's errand. You could put all of the perfect elements together in a tightly constructed piece and it could still fail. Really, you're likely far better off focusing on visibility for your work than you are focusing on building the perfect mousetrap.

Okay, so maybe you can point to certain trends or cliches that are common denominators in those stories that turn out to be hits, and maybe you can string some of those together and wind up with what one might consider a more 'commercially viable' product than something that doesn't contain many/all of those trends or cliches...but, for every one writer who manages to succeed by doing so, there's five others who've followed the same path and wound up nowhere. Why? Because again, the whole thing is squishy, foggy, unknowable, i.e. human. We're all making guesses and more often than not we wind up wrong.

No one in Hollywood sets out to make a dud, and I imagine the same is true in publishing. Everyone, if they had their way, would create something that truly resonates with people and make all the money, but knowing what's required to do that ahead of time is fuzzy at best. All you can really do is put your heart and soul into your creation, work and spend to make it as visible as possible, and then hope it takes off. That's kinda scary and uninspiring, and there's not really any money in selling that message to all the bestseller-wannabes out there, but I think it's the truth.

And really, everything put forward as being a means to 'crack the code' or discern the 'formula' or 'unleash the power of the algorithms' is actually just thinly veiled coping mechanisms for the many who don't want to deal with the intimidating un-calculable squishiness of consumer tastes, or maybe it's a message forwarded by those who don't want to come to grips with the fact that they're eventually going to have to spend on advertising if they're to have a snowball's chance...OR, it's the recognition by some of the more cynical (or savvy?) among us that there's more money to be made in selling the idea of predictability than there is in actually trying to write the next big bestseller. Kind of like how some geniuses out there realized there's more to be made in selling the ball bearings than there is selling the tanks.
 
The following users thanked this post: Tom Wood

Post-Doctorate D

Re: Quartz Article - Algorithm-Driven Fiction
« Reply #2 on: November 21, 2018, 07:53:30 AM »
The keyword is potential.  The algorithms can only predict potential popularity.

It's not that they are saying follow this formula and your book will be popular but rather that this is the formula that popular books follow.  It doesn't mean that if you don't follow the formula you will be unsuccessful, but rather that if you don't follow the formula you will have a smaller chance of being successful.  Likewise, if you follow the formula you have a higher chance of writing a popular book but following the formula doesn't guarantee that you will.

So the algorithms serve to minimize risk and maximize potential.  It doesn't guarantee success only a higher likelihood of success.

It's like saying if you write romance you will have a higher probability of becoming a full-time author than if you write poetry.  Sure, you could become a full-time poet, but the odds are very slim compared to becoming a full-time romance author.
"To err is human but to really foul things up requires AI."
 
The following users thanked this post: Tom Wood

Tom Wood

Re: Quartz Article - Algorithm-Driven Fiction
« Reply #3 on: November 21, 2018, 08:16:20 AM »
Dan - Yes, exactly.

The authors of The Bestseller Code offer a manuscript analysis service that uses their comparative method. For $200 they will score and compare your book to the average score of the bestsellers in your genre.

http://www.archerjockers.com/services/single-manuscript

I wonder if the authors who publish under the Amazon imprints ever get notes from the robot.

Raylan - Thank you for that thoughtful response. I'm not endorsing the computerized analysis, I just thought it was an interesting part of the crazy world we live in.
« Last Edit: November 21, 2018, 08:29:53 AM by Tom Wood »
 

CoraBuhlert

Re: Quartz Article - Algorithm-Driven Fiction
« Reply #4 on: November 21, 2018, 09:58:28 AM »
I suppose what I'm saying is, the whole thing is squishy, for lack of a better term. People are strange and unpredictable. Tastes vary and can turn on a dime. I've been reading books that I've loved and found fantastically gripping and yet I've put them down and walked away in the middle of particularly engrossing passages. Why? Who knows? Maybe my cold made me drowsy. Maybe a friend arrived. Maybe something distracted me. Maybe something bothered me from 50 pages before and wore at the back of my brain until I felt compelled to stop and think on it. Maybe I smelled and needed a shower right then and there. Maybe my oven was finished preheating. That's just it. There's a million and one reasons why people do this while not doing that. Why they put a book down or pause an audiobook or speed it up or do pretty much anything. Of course, my examples are purely anecdotal, but I doubt they're all that unique. I'm sure I'm not the first to stop consuming something they were enjoying and then forget to go back to it, or they go back to it for any given reason at any given time.

Yes, this. For example, last year I was reading a book I enjoyed a whole lot. Fairly close to the end, there was a medical emergency in my family, so I put the book aside and didn't pick it up again for a whole month. Going purely by Amazon data, the book didn't grip me, even though it did. Because the algorithm has no way of telling why a book was put down or abandoned.

Blog | Pegasus Pulp | Newsletter | Author Central | Twitter | Instagram
Genres: All of them, but mostly science fiction and mystery/crime
 

guest957

  • Guest
Re: Quartz Article - Algorithm-Driven Fiction
« Reply #5 on: November 21, 2018, 10:37:15 AM »
The keyword is potential.  The algorithms can only predict potential popularity.

It's not that they are saying follow this formula and your book will be popular but rather that this is the formula that popular books follow.  It doesn't mean that if you don't follow the formula you will be unsuccessful, but rather that if you don't follow the formula you will have a smaller chance of being successful.  Likewise, if you follow the formula you have a higher chance of writing a popular book but following the formula doesn't guarantee that you will.

So the algorithms serve to minimize risk and maximize potential.  It doesn't guarantee success only a higher likelihood of success.

It's like saying if you write romance you will have a higher probability of becoming a full-time author than if you write poetry.  Sure, you could become a full-time poet, but the odds are very slim compared to becoming a full-time romance author.

This is insightful. Or is that, inciteful?  :ices_angel_g:

No, seriously, I think you're right to make that distinction. It's something that perhaps I hadn't fully allowed for in my post. While I do think it folly to put too much stock in what the codes and algorithms say, I do recognize the logic inherent in your romance/poetry analogy.

Then again, I attribute that more to identifying the shape of a building's steel frame as opposed to trying to game out the floor plan and the arrangement of the furniture and selling that to others as the granular path one must follow to find financial success. Identifying the shape of the frame seems useful, but I'm not so sure about delving deeper than that because of the squishiness I'd alluded to in my previous post. I worry that aspiring authors out there are paying for said granularity - not that I blame them - it's a tough business. But chasing that particular white rabbit will likely only lead to frustration and lost time/money.

I'm not so certain Amazon and Netflix are looking at their deep pools of data and thinking they're only good for seeing the forest for the trees or rather only the potential, I think they likely believe they'll be able to tease out the genus of the lichens growing on each individual bark as well, and that they'll be able to reverse engineer "hits" from the very DNA buried deep within. To me, that way lies madness. And, unfortunately, I think there are "teachers" out there running with that same logic in the writing world who sell "systems" built not just around telling aspiring authors what could potentially work, but what will work in creating a bestseller down to the level of a quark, when really said aspiring author's time/money would likely be better spent honing their visibility chops or putting their whole selves into the stories they're telling every day, rather than enrich the ball bearing salesperson who's discovered there's gold in them thar desperate hordes of bestseller wannabes.

And now that I've beaten every analogy to within an inch of their lives and mixed every metaphor from here to Saturn, I should likely stop typing.  :hehe
 

guest957

  • Guest
Re: Quartz Article - Algorithm-Driven Fiction
« Reply #6 on: November 21, 2018, 10:40:47 AM »

Raylan - Thank you for that thoughtful response. I'm not endorsing the computerized analysis, I just thought it was an interesting part of the crazy world we live in.

No worries, I didn't take your original post as an endorsement at all. I'm glad you posted the article and started the discussion. It's all really interesting stuff.  :icon_cool: