Comments for Do recommendation systems make the ‘tail’ longer or shorter?

As an introduction, I will remind the long tail theory.
In the chart (1), we can clearly see what is the long tail: Seen by Chris Anderson, the long tail includes the products that are less popular than the head. The fact is that the popular products are sold in large quantities where the niche products are sold in low quantities but consist of a large variety of products. The long tail theory says that the (niche) products in the long tail outnumber those ones in the head. (1)
Indeed, the Long Tail theory asserts that “the products with a low demand, or a low sales volume can collectively make up a market share that rivals or exceeds the relatively few current bestsellers and blockbusters, but only if the store or distribution channel is large enough.”. (2)

It seems to me that this long tail can be a potential market share that Internet made available with the fall of the costs of production and distribution.
As explained in the comment, the long tail is becoming thicker (and longer) as the niche products are doing better in the digital age and become more popular. Hence, those niche products are interesting and big retailers understood it.
A little example may illustrate the long tail theory: An Amazon employee said that “We sold more books today that didn’t sell at all yesterday than we sold today of all the books that did sell yesterday.” (3)

After this remind, we can ask: How the recommendation systems impact the long tail and the niche products?
There exist more and more recommendation systems. “The top more popular in this category” but also “People who bought this also bought…”. They appear on the big retailer website and on the mobile platform such as Google Play and Apple. That can be a problem because the recommendation systems are based on the sales and rating. Hence, those systems will redirect the customer to popular products again and again. As said in this article: “Because common recommendation systems are based on sales and ratings — for example, people who bought this also bought this — they’re unable to surface truly novel items that have not been discovered by many other people.”. (4)
I think that as long as customers will use the recommendation systems based on sales and ratings, they won’t likely discover niche products, as explained in this article: “This tends to create a “rich gets richer” effect for popular items, and it might also prevent consumers from finding better product matches because of this bias for items that have been purchased by others or that have been rated well by others.”.
However, a little further in the article, they made two conclusions: the first one is that however, the recommendation systems were supposed to help the niche products to obtain opportunities, recommendations systems doesn’t necessarily help to discover niche products
The other conclusion is that when a product with low rating is recommended, people may give the benefit of the doubt and give them more attention. (4)

References:
(1) http://www.longtail.com/about.html
(2) http://www.investopedia.com/terms/l/long-tail.asp
(3) http://longtail.typepad.com/the_long_tail/2005/01/definitions_fin.html : Third comment
(4) http://knowledge.wharton.upenn.edu/article/recommended-for-you-how-well-does-personalized-marketing-work/ (December 2015)

Let’s start this case analysis by trying to make a clear definition of the longer tail “effect”: If all the goods from a market were normally distributed, the tail of the distribution will represent a niche market (sells unique goods in small quantities) when the center of the distribution will represent the main products (blockbuster, best-seller, …). In other words, the well-known products have a more powerful sales-force than the unknown-one: logic. But the interesting thing is that with the internet revolution and indirectly, the e-commerce, companies are not physically constrained anymore: they can put in plain view all the products they want. It doesn’t sound that significant, but back in the time, a bookseller for example, has no interest to a book which has not any guarantee of success in his store. Indeed, it is most likely that a very small amount people will buy it. Especially when the last book of a famous writer just came out. As he is physically constrained (limited capacity), he is more or less obliged to only propose mainstream books. This small example allows us to realize some important effects.

In today digital world, anyone can put anything anywhere on the internet and this includes big companies like Netflix and Amazon. They transformed this new power in a marketing tool which allows them to adapt the content of a website page according to the user’s interest (recommendation system). Therefore, nowadays, they are able to offer us a larger ranged of products, including the less known one which nobody is aware of. Economically speaking, on one side, consumers will be satisfied as they discover new products which are potentially going to interest them, and on the other side, selling these products is going to be more profitable for the niche market producers. Both consumer and producer welfare is enhanced. To me, it is definitely something crucial and important. This effect is lowering the normal distribution and rebalance the situation.

Furthermore, the recommendation system (personalized marketing) has evolved with an extension to the initial concept: the online reviews. The figures speak for themselves: 90% of consumers read online review before visiting a business, 88% of consumers trust online reviews as much as personal recommendations. Here again, we can see the strength of a personal recommendation system as a personalized marketing tool.

To conclude this discussion about the recommendation system, we can see that internet and the globalisation are making the long tail taller, increasing the numbers of available products on the market. Personally I think that this will progress again and again, and that marketers will never stop to innovate, in a good way I hope. This is a very interesting subject which is complex and has more impact than just a variation in quantity/price. I mean, we can question ourselves about the respect of privacy of companies using this tracking techniques. Most people using internet don’t even know that their data are collected. Anyway, it has a significant positive impact on consumers/supplier. In my opinion, I think this is a step forward for a better balanced economy where all economic players have more or less the same success chance. Ultimately, this further strengthens the competition instead that this time, it’s not just a harder fight between the main actors of the market…

References:

Su, N., Levina, N., & Ross, J. W. (2016). The long-tail strategy of IT outsourcing. MIT Sloan Management Review, 57(2), 81-89.

http://knowledge.wharton.upenn.edu/article/recommended-for-you-how-well-does-personalized-marketing-work/
http://www.business2community.com/infographics/impact-online-reviews-customers-buying-decisions-infographic-01280945#8IAPzyX6JW6rg88v.97
http://www.cs.umd.edu/~samir/498/Amazon-Recommendations.pdf
http://www.marketing-schools.org/types-of-marketing/long-tail-marketing.htmlV

Introduction

Recommendation systems « identify recommendations autonomously for individual users based on past purchases and searches, and on other users’ behavior ». (1) These systems have become more and more common in recent years and I often face with these one while I’m visiting a panel of different applications such like music on Spotify, books on Amazon or movies on Netflix.
Toward the realization of this comment, I have focused myself on the music application called Spotify I’m using everyday in order to understand the evolution of the recommendation system and how this technique of recommendations could make the tail longer or shorter.

Collaborative filtering system

Spotify recommender system was mainly based on collaborative filtering techniques. These approaches are defined as follows « collaborative filtering is the process of filtering for information or patterns using techniques involving collaboration among multiple agents, viewpoints, data sources, etc. » (2) In other words, « the idea of collaborative filtering is to determine the users’ preferences from historical usage data » (3). For example if two users listen to largely the same set of songs, their tastes are probably similar.

Limits of the system

But this system seems to have certain limits. In fact, two main issues called « content-agnostic » problem and the « cold-start » problem were encountered by Spotify. The first one is met because the technique of collaborative filtering don’t use any kind of information about the items that are being recommended. Given this, « recommendation systems become predictable and boring in the sense that popular items will be much easier to recommend than unpopular items, as there is more usage data available for them ». (4) The second one, called « cold-start » problem, is encountered when songs are new or unpopular. In fact, in this case, there is no data to analyze and then the collaborative filtering system breaks down : recommendations don’t work for these songs. A last problem from the collaborative filtering system, which can be mentioned here, is the heterogeneity of content with similar usage patterns. In fact, when I’m listening a playlist or the entire album from an artist, these one may contain intro tracks, cover songs and remixes. The collaborative filtering algorithms are only coded to take the original song and won’t take all other forms of songs to draw recommendations.

The solution called « Deep Learning »

In order to solve these different issues, Spotify has had to take different actions to incorporate other sources of information into its recommendation system. Spotify found another approach called « the content-based recommendation ». In this system, the firm is taking other sources of information as tags, lyrics, text mined from the web (reviews, interviews,…) or the audio signal itself for example.
It’s precisely this last source of information (the audio signal) that Spotify began to analyze in order to predict what songs users will enjoy. The algorithm created by Spotify is able now to learn the specific content of the songs following different factors as specific pitches, chords, sounds, and others. With this, filters are set up to cluster songs in different categories following these last factors. For instance, if a new song comes on the platform or if this song comes from an artist that’s not popular, it would be hard to recommend that to users if Spotify is using more traditional methods because no one else is listening this song. But with this new method, the algorithm is relying on the acoustics of the song (for example), it can make better recommendations (lesser-known artists i.e.) for users.

Conclusion

As a conclusion, people in the past were discovering products which were already rather popular in the whole population. However, Spotify managed to overcome and resolve this problem. In fact, the firm created a new algorithm which allows them to recommend new and unpopular musics what wasn’t possible with collaborative filtering systems.
In general and based on the Spotify case, we can see that firms are in the process of finding systems which are able to recommend new products or unknown products to customers. In the past, one would have thought that recommendations systems were making the tail shorter. But in reality and thanks to new technologies, we see that the tail is becoming more and more longer…

References

(1) http://www.ibm.com/developerworks/library/os-recommender1/
(2) https://en.wikipedia.org/wiki/Collaborative_filtering
(3) (4) http://benanne.github.io/2014/08/05/spotify-cnns.html
– http://www.wired.com/2004/10/tail/
– http://www.webopedia.com/TERM/R/recommender_systems.html
– https://www.washingtonpost.com/news/the-switch/wp/2016/03/11/spotify-might-be-secretly-stealing-your-taste-in-music-and-pitching-it-to-other-people/
– https://gigaom.com/2014/08/05/how-spotify-is-working-on-deep-learning-to-improve-playlists/
– http://thefuturesagency.com/2015/08/01/spotify-using-deep-learning-to-create-the-ultimate-personalised-playlist/
– http://observer.com/2015/05/spotifys-head-of-deep-learning-reveals-how-ai-is-changing-the-music-industry/
– https://www.macstories.net/linked/recommending-music-on-spotify-with-deep-learning/
– http://qz.com/571007/the-magic-that-makes-spotifys-discover-weekly-playlists-so-damn-good/
– http://fr.slideshare.net/MrChrisJohnson/interactive-recommender-systems-with-netflix-and-spotify/94-Step_4Radio_thumbs_feed_back
– http://dataconomy.com/spotify-used-deep-learning-recommend-songs-youll-love-never-knew-existed/

Recommendations systems are increasingly present on many websites. We particularly cited non-exhaustively the most famous like Amazon, Netflix, Pandora, etc.

This topic is quite complex and involves many mathematical concepts and probability. I think that the data collect is one of the important topics to discuss when we talk about the recommendation system. The data collection can be done in two ways: implicit and explicit. Implicit data collect is rarely used for websites because it is more binding on the consumer. Indeed, the consumer must explicitly show these preferences in relation to a set of products or services. This may for example be through a ranking system where the user class products / services based on these preferences.

The implicit collect of data, meanwhile, is more complicated to implement because it uses more complicated algorithm but is much less restrictive over the consumers. It is this type of collect is most often used for websites. This may for example be based on the consumer objects seen on the website. Other criteria, less common, such as interests, the places of residence, the age of the consumer, etc. can also be took into account.

Of course, each of the data input to the algorithm must make a difference in importance according to criteria defined by the website. Example, we can easily imagine that the data “last purchase made” or “product in the basket” have far more important than ata “product page visited.”

To maximize the effectiveness of recommendation systems, a latent semantic models is often used. Professor Saerens said in his courses: “We say that semantically related words are used when they are in the same context “. For example, the words “baby” and “newborn” have semantic similarities. This allows, for example, that the recommendation system provides consumers with product recommendation with description as “baby pushchair” while the consumer was looking for “newborn pushchair.”

Source :
http://fr.wikipedia.org/wiki/Système_de_recommandation
http://www.cs.umd.edu/~samir/498/Amazon-Recommendations.pdf
Slides of the professor Saerens for the course of Data Minning (UCL, actuarial curriculum)

When the Internet was first created, its main purpose was to serve as a platform of information that people could just as well inject as collect. With the years passing by, the quantity of information the web contained grew tremendously.
Nowadays, it is possible to find thousands of answers to a simple question. In my opinion, we have reached a level of knowledge so high that the efficiency of the search engines such as Google, Yahoo, etc. has been compromised. Bertram Gross defined the phenomenon of “ information overload”, in his book ” The Managing of Organizations”, the following way: Information overload (also known as infobesity or infoxication) refers to the difficulty a person can have understanding an issue and making decisions that can be caused by the presence of too much information.

Recommendation engines are therefore very useful tools, since they scan and classify the billions of data according to the desires and preferences of each individual.
However, it is not relevant to advertise products that may have already been easily discovered by the user, being famous products falling in the taste category of the consumer. A popular product does not need a boost to be visible to the consumer.
It can even be annoying to perceive the obvious individual advertising on some websites.
I believe the power and value of a recommendation system lies in its ability to introduce us to new things. Products that suit our interests, but that we would’ve never discovered without the recommendation data research engine digging it up for us. Indeed, it results in giving us the impression of finding a rare unknown pearl among the other products. 


Therefore, a solution to make the tail longer could be to sensitize and boost the algorithms of recommendations systems. In order to accentuate their research performances and increase their capacity to dig deeper in the giant ocean of information the Internet is today, and brings new products to the consumers.

As conclusion, I think it is important to promote the application of the long tail theory. If we are using Internet platforms to purchase objects, it is to have access to a range of varied personalized proposals and not to be found in the case of a local business.

References:

http://en.wikipedia.org/wiki/Information_overload#cite_note-1
http://www.podcastscience.fm/dossiers/2012/04/25/les-algorithmes-de-recommandation/
Gross, Bertram M. (1964). The Managing of Organizations: The Administrative Struggle. p. 856

First of all, the recommendation system ables to “predict the rating or preference that user would give to an item thanks to the user’s past behaviour” (Wikipedia). This system permits to reveal the truth about what consumers want and in some cases, helps some items to be selled again (Touching the Void of Joe Simpson with his book that increases their sales decades after his release). But is that recommendation system increases the long tail effect ?

I think it depends on the sector, the type of the product and the algorithm used by the website. I’ll take some example. If you use an algorithm which recommends you a book or a film with the same subject it could be very interesting and it makes the tail longer. With « Allocine », when you read details and the test of a film, the website provides other films that « you could like » on the same subject (and not only best-sellers or films from the same producer). From my own experience, I really like film of « prisons ». When I read the test of « The Green Mile », I discovered a lot of really interesting films on this subject and which was not really known by people because of the lower marketing resources of the producers.
On the contrary, Youtube recommends most of the time « videos » or « music clips » that are the most viewed (« Vevo » recommebds each time the most popular songs). We also realize that some « videos » are all the time recommended. Youtube favors the biggest creators with more money who pay to be seen. In this case, recommendation make the tail shorter for the small budgets.

In order to improve his recommendation system, Netflix launch a competition « the Netflix Prize », where the best improvement was rewarded of 1000000$. The winner of this competition improved the algorithm just 10% better but finally, they never implemented this solution in their company. WHY ??
Because the « additional accuracy gains that we measured did not seem to justify the engineering effort needed to bring them into a production environment » (Forbes.com).

To conclude, the approach of your recommender system will depends on your algorithm and what you want for your website. The first approach is the Collaborative filtering “which can be constructed from a single user’s behavior or also from the behavior of other users who have a similar behavior.” The second one approach is the “Content-based filtering which constructs a recommendation on the basis of a user’s behavior” (ex : historical browsing information). You also have hybrid approaches which is a mix of these both approaches and which could be the best thing according to your personal offer.

Sources :
http://www.ibm.com/developerworks/library/os-recommender1/
http://en.wikipedia.org/wiki/Recommender_system
http://archive.wired.com/wired/archive/12.10/tail.html
http://en.wikipedia.org/wiki/Netflix_Prize
http://www.forbes.com/sites/ryanholiday/2012/04/16/what-the-failed-1m-netflix-prize-tells-us-about-business-advice/

For many platforms recommender systems play a role as important as the Search, Advertising and Networking systems; for example, for Google News recommendations generate 38% more clickthrough, 35% of Amazons sales comes from recommendations, 2/3 of the movies watched at Netflix are recommended and 28% of the people at Choicestream would buy more music if they found what they liked. As available data is become bigger and bigger the algorithms used to build Recommender systems are becoming more complex and accurate meaning that the importance of user and data mining is going to be a growing trend in many areas for such us platforms.

According to “Recommender Systems Handbook” authors, there are 2 “traditional” recommender systems: Collaborative Filtering (recommend items based only on the users past behavior ) which can be user-based if the recommendations depend on what similar users to me prefer or item-based if they depend on similar items to those that I have previously liked. The second one is Content-based Recommendations (based on items features).
There are also some “novel” methods including Personalized Learning to Rank (treating recommendation as a ranking problem), Context-aware Recommendations (recommendations based on user features, specially demographics) and Social recommendations based mostly on users similarity and trust. What most platforms put in practice is a Hybrid method that combines any of the above methodologies depending on the domain and particular problem the platform is face to or in order to improve results in specific cases such as the “cold-start” problem those platforms face when they don’t have yet an important users base. The most common Hybrid methods are Weighted where outputs from several techniques (in the form of scores or votes) are combined with different degrees of importance to offer final recommendations, Switching (depending on situation, the system changes from one technique to another), Mixed (several techniques at the same time), Feature Combination (features from different recommendation sources are combined as input to a single technique), Cascade (the output from one technique is used as input of another that refines the result), Feature augmentation (The output from one technique is used as input features for another) and Meta-level where the model learned by one recommender is used as input to another. According to Amatrianin (2014), in the general case it has been demonstrated that the best isolated approach is Collaborative Filtering.

As a practical example. In Netflix recommendation approach they go from the rating system (main input) with which they build a ranking and finally a “page generation” (grid with rows and column). Everything is personalized, for example the genre rows are generated in an implicit way (based on user´s recent plays, ratings, and other interactions), explicit way (taste preferences) and combining the both (hybrid way). Thanks to their methodology the go from a “page generation” of 10,000´s of possible rows and thousands of possible videos for each row to one personalized page of 10-4o rows. They try to have a balance between accuracy and diversity, discovery and continuation, depth and coverage and Freshness and stability; this is in line with the 3 rules described in The Long Tail, especially when having a balance between Hits and less well-known films.
From the Netflix case is interesting how they don’t put so much emphasis on the social side since they argue that if they know enough about their customers, social information becomes less useful; they track what users get shown and their interactions but they recognize that from an infrastructure perspective it is very costly. Also regarding the context they had to improve their recommender systems, Netflix own offline studies showed the wining models were too computationally intensive to scale so the expected improvement was not worth the engineering effort (they also acknowledged that their focus had shifted to other issues that had more impact than rating prediction).

Sources:
“Recommender Systems Handbook.” Ricci, Francesco, Lior Rokach, Bracha Shapira, and Paul B. Kantor. (2010). http://www.cs.bme.hu/nagyadat/Recommender_systems_handbook.pdf
http://www.slideshare.net/xamat/recommender-systems-machine-learning-summer-school-2014-cmu
http://misrc.umn.edu/wise/2014_Papers/134.pdf
http://en.wikipedia.org/wiki/Long_tail
http://archive.wired.com/wired/archive/12.10/tail.html?pg=3&topic=tail&topic_set=

According to Chris Anderson « The theory of the Long Tail is that our culture and economy is increasingly shifting away from a focus on a relatively small number of “hits” (mainstream products and markets) at the head of the demand curve and toward a huge number of niches in the tail. As the costs of production and distribution fall, especially online, there is now less need to lump products and consumers into one-size-fits-all containers. In an era without the constraints of physical shelf space and other bottlenecks of distribution, narrowly targeted goods and services can be as economically attractive as mainstream fare »(1)

In my opinion, recommendation systems can in certain cases make the “tail” longer but it depends on certain factors.
In order to understand how online product recommendations could influence consumers’ online choices, we should refer to the article of Sylvain Senecal and Jacques Nantel (2).
In their article, they aim to investigate consumers’ usage of online recommendation sources and their influence on online product choices.
In their study, they focus on three determinants that could influence the impact of computer-mediated recommendations on consumers’ online product choices.
The three determinant that they investigate in their study are: the nature of the product recommended, the nature of the website they are recommended on and the type of recommendation source.
The first determinant underlines the fact that goods can be classified as possessing either search qualities (those that the consumer can determine by inspection prior to purchase) or experience qualities (those that are not determined prior to purchase).
The second determinant underlines the fact that the nature of the website can also influence the impact of a given recommendation.
They are three kinds of websites: sellers, commercially linked third parties and non-commercially linked third parties.
The last determinant sorts the type of recommendation sources in three categories: other consumers, human experts and expert systems.
After testing several hypothesis on these assumptions, they were finally able to show that consumers would be more influenced by recommendations for experience products than for search products which seems logical as it is difficult or even impossible to evaluate experience products before purchase, consumers should then rely more on product recommendations for these products than for search products.
But also that no relationship exists between the type of website and subjects’ propensity to follow product recommendations
And finally, that the recommender system (expert system) is more influential than other consumers and human experts.
It appears that a recommendation source providing personalized information to consumers is more influential than a recommendation source providing non-personalized information.

To sump up, I think that the article of Senecal and Nantel help us understand that the theory of the long tail is not sufficient to itself and that other factors must be taken into account if we want to evaluate the influence of a recommendation system.
Indeed, I think that a recommendation system could make the tail longer but it will mostly depend of the type of product and the type of recommendation system.
Having said that, I think that the theory of the long tail doesn’t apply with the same intensity in every sectors or industries.
Then, we cannot generalize a positive impact of the recommendation systems.

(1) http://www.longtail.com/about.html
(2) Senecal,S., Nantel,J. (2004). The influence of online product recommendations on consumers ‘online choices, Journal of retailing, 80, 159-169.
(3) http://archive.wired.com/wired/archive/12.10/tail.html?pg=5&topic=tail&topic_set=

A first update worth mentioning is the distinction between user recommendations and system recommendations. This distinction is around for a wile now, however, only recently these different types are empirically investigated, moreover the difference in effects of both was studied.

On the one hand we have system recommendations, which are most commonly generated by the collection of all user preferences. With these preferences the firm will suggest products to consumers. A very well known example of a system recommendation is the co-purchase recommendation of Amazon. Here, recommended products will be listed as “customers who bought this item also bought”.

On the other hand we have the user recommendations or also called user-generated word-of-mouth (WOM). They display consumer reviews of products whereby the review is based on the volume and the rating of the product.

This division of recommendation systems into two different types is necessary according to the paper of Zhijie Lin because according to her, both recommendations will generate different impacts. According to Lin, user recommendations are more effective than system recommendations in driving product sales.

Recommendation systems are already very popular and in my opinion they will only increase during the years. I based this comment on some striking results I found in a recently conducted survey:
– 85% of customers consuls online reviews before purchasing
– 65% of consumers claim that there purchasing decision is influenced by online reviews of other customers.

To further optimize the recommender systems firms need to increase the amount of consumer feedback that they receive. This because customer feedback will improve accuracy and can lead to a learning curve for the firm to notice a pattern in consumer preferences.

Literature has extended its knowledge about the efficacy of the recommendation systems quite a lot. Literature found that there are some differences in efficacy in relation to the sector/ type of products sold but also in relation to the characteristics of beneficiaries and in firms.

A recent paper from Harvas-Drane (2015) found empirical prove for the fact that recommendation systems do not work as efficient in all sectors. He argues that these recommendation systems are very useful for the following products: music, films, books and video and game entertainment.

He also makes a distinction in efficacy of the recommendations according to the characteristics of beneficiaries. According to him, customers with less prevalent preferences will create a long tail effect and the concentration of sales will be reduced. Consequently products that have a smaller share of total sales, otherwise called niche products, will generate a long tail. This finding is still in accordance with the long tail theory, firstly discussed by Anderson in 2004.

The firms that have lower inventory costs will benefit the most of the long tail effect because they can increase the depth of their assortment more easily and to a wider extent then others. As a consequence these firms are better positioned to serve demand for niche products (that leads to the long tail effect).

A recent study argues that word of mouth can contribute to the explanation of the long tail phenomenon.

Harvas-Drane (2015) links recommendation systems with word of mouth, which led to some interesting findings. He argues that recommendation systems and word of mouth could benefit all consumers (because of the lower search costs), but they have a larger impact on those consumers that have less prevalent preferences. However, there are also benefits for the firms: the market share of niche products in the sales distribution will increase, leading to a longer tail. But also a reduced concentration of sales within the firm’s assortment is noted.

References:

Hervas-Drane, A. (2015). Recommended for you: The effect of word of mouth on sales concentration. International Journal of Research in Marketing .

Zhijie, L. (2014). An empirical investigation of user and system recommendations in e-commerce. Decision Support Systems (68), 111-124.

I have notice that we can find a lot of things on the long tail phenomenon and the recommendation systems. The first paper I would like to mention has been written by Bin Gu, Qian Tang and Andrew B. Whinston and deals with the effect of online information on the long tail phenomenon. In this paper the authors mentions two opposite beliefs concerning the effect of information on the formation of long tail. The first belief considers that the existence of online information available for customers encourages the formation of long tail. On the other hand, the second belief thinks that the customers ignore the available information and that they base their choices only on their personal beliefs.
The three authors have tried to determine which belief is the right one. Their study showed that positive comments improve the sale of popular product more than the sale of niche product. Consequently it means that the existence of information on the Internet has a negative impact on the formation of long tail.
I find this conclusion interesting because personally I always read comments of other customers before watching a film for example. Moreover I think those comments have more and more importance on the web. Indeed many people decide to make comments and therefore their influence should be increasing.

Huanxing Yang has developed in 2013 the second analysis I would like to mention. He has created a search model to explain the long tail effect. In his theory, the long tail effect is due to the decrease in the price of searching or an increase in the search targetability. What does the author means by the price of searching and search targetability? Today with the Internet, customers have easily access to a large number of products. They are note limited to the products available in the store they visit. Consequently the distribution of sales has become flatter and niche products have gain market share. So the long tail effect is encouraged by the decrease in the price of searching and the increase in the search targetability.

Finally I have noticed that several techniques have been developed in order to maximize the formation of long tail. Techniques such as adaptive clustering, hybrid music recommender system which combined usage and content data, ect.

References:

Jeyshirii, S., Thivakaran,T.K. (2014). Precise Recommendation System for the Long Tail Problem using Adaptive Custering Technique. International Journal of Innovation Research in Computer and Communication Engineering, 2(4).

Gu, B., Tang, Q., & Whinston, A. (2013). The influence of online word-of-mouth on long tail formation. Decision Support Systems, 52, 474-481.

Yang, H. (2013). Targeted search and the long tail effect. RAND Journal of Economics, 44(4), 733-756.

Firstly we have to say that the recommendation system is a very useful tool to help the customer for finding the genre(s) and style of products they like. But it has to be used effectively. Indeed, like said in the article this concept of recommendation system is not new anymore and a majority of customers rely mainly of this kind of system in order to find other product related to their taste.

The fact is that the utilisation of this system is so fast with internet now that it will be (for a lot of persons) a waste of time to do research of their own. We can point the example of ebay,alibaba or even amazon like it was presented in the text. The recommendation system in the site are just under the image of the command and then really easy and fast to find.
However eventough the recommandation systems have their avandtages we can easily point out bad consequences of those kind of processes.

we can explain those consequences in 2 main points.
The first one is what we call “collaborative filtering”. This represents the process of filtering information by using techniques involving collaboration among multiple agents. The main point is to help the customer to find popular films, musics, books or whatsoever related to their taste. The fact is that this system doesn’t take into account a lot of works made by newcomers or simply unadvertised works. Indeed the fact is that the customers can only see 5 or 6 other works in this kind of recommendation system, this increases then the inequality factor between those kind of works.

The second point is called “collective intelligence”: which are the cognitive abilities coming from the interactions in a community between the members. This knowledge of those members is then limited because their vision is highly restricted due to the fact that they share the same vision of the environment (world) and are then not fully conscious of the surroundings.
We can apply this to the case because a lot of recommendation system create this kind of community and then close the possibility of the members to expand their vision in the genre they like.
We can then conclude that only using recommendation system will favor only few works and then don’t give the chance to a lot of potentially good contributions. The only use of recommendation system will then create a shorter tail for the market with a huge difference between blockbusters and other works.

sources:
http://en.wikipedia.org/wiki/Recommender_system
http://en.wikipedia.org/wiki/Collective_intelligence
http://www.businessdictionary.com/definition/collaborative-filtering.html
http://en.wikipedia.org/wiki/Collaborative_filtering

According to Anderson, our economy has shifted from a small number of highly popular products (the head) towards a high number of niche products (the tail). Indeed, in traditional economies, popular products were favored in comparison to niche products because the physical room on the shelves was limited and expensive. With the internet revolution, it has become in the contrary easier to build a business on the long tail given that inventories are not limited and that niche products are easier to find.

In his book, Anderson explains that 2 conditions lead to successful long tail business: making everything available and helping the customers to find the niche product. The second condition is fulfilled with the recommendation tools develop by players in long tail business. Recommendation systems design aims at making the consumer select the most important item for him.

What is at stake in this post is the effect of recommendation systems on sales. I will consider the 2 issues that are the most present in the literature: the impact on sales volume and sales diversity. Diversity is directly linked to the long tail phenomenon. Indeed, recommendation systems can have two distinct impacts:

 Helping consumers discover new products that fit better with their preferences, hence increasing the diversity;
 Reinforcing the popularity of already famous products, hence leading to more concentrate sales and thus less diversity.

It had been proved that recommendation systems have a significant impact on sales volume but the impact on the diversity of purchase seems to be more controversial.
This post was originally written in 2012 and I would like to update it with a paper written in the context of the 35th international conference on information systems in 2014 by Lee and Hosanagar.

In this paper, the authors analyze the impact of different recommender algorithm on both sales and diversity since, according to the authors, different algorithms have different impact on these two variables. The debate in the literature on the effect on diversity by recommenders systems is due to these differences. Recommendation systems are classified by Schafer, Konstan and Riedi in 6 types: the collaborative filtering (consumers who bought this item also bought that item), the content-based (according to your consumption history), the hybrid (combining the first two types), the social-network (what your friends bought), the popularity recommender (the most purchased items) and the recently-viewed (the most viewed items). Lee and Hosanagar analyze the impact of different recommender systems by measuring the diversity at the item and genre level with the use of the Gini coefficient mentioned in this post. The Gini coefficient has a range of values between 0 and 1 and measures the level of concentration diversity. A coefficient of 0 means that all products have the same amount of sales, while a coefficient of 1 means that a number of popular products account for most of the sales.

As explained before, the kind of algorithm used has an influence on sales volume and diversity. I am going to compare two of them: the collaborative filtering and the recently-viewed algorithm. One of the results of the study is that a collaborative filtering algorithm increases the sales volume and the individual diversity but decreases the overall aggregate diversity. Indeed, it seems that individuals discover a higher diversity of products through links provided to other items that can match with their preferences. However, the algorithm introduces a popularity bias since consumers move towards the popular items rather than towards niche products, which introduces a concentration in the diversity at the aggregate level, and so making the tail shorter. As far as the recently-viewed algorithm is concerned, it decreases the aggregate diversity and doesn’t influence the individual diversity. In fact, this algorithm, by showing you the recently viewed items, decreases the views on other less popular items and so decreases the diversity.

To conclude, in this comment, I have analyzed some issues related to niche and popular products and the influence of recommendations on them. However, one further investigation could be to analyze to what extent the previous analysis applies to products that are classified as neither popular nor niche item.

References:

Anderson, C. (2006) The Long Tail: Why the Future of Business is Selling Less of More. Hyperion.

Schafer, J. B., Konstan, J., & Riedi. (1999). Recommender systems in e-commerce. Proceedings of the 1st ACM conference on Electronic commerce, 158–166.

Lee, D., & Hosanagar, K., (2014) Impact of Recommender Systems on sales volume and diversity, 35th international conference on Information Systems.

Shanthi, K., & Kalimuthu, M., (2013) Improving diversity using contend based approach in recommender systems, Research Journal of Computer Systems Eengineering, 4.

My comment will be structured in three parts. I will first give my opinion about long tail theory, then about the recommendations systems and I will finish with the relation between the two previous.

From my opinion, the long tail theory doesn’t apply to every sectors or at least it doesn’t apply with the same intensity. In some industries, it’s true or more true that a large part of selling products distribution came from a large number of products each sold in low quantity. As Chris Anderson show it with the examples of Amazon and Netflix. I think that it’s also apply in the book sector more than in the music or film sectors. I see two distinct reasons. Firstly, production studios as Warner or Universal exert a massive brain-washing and they promote a lot a few products. They have a larger impact on people than publishing houses. Secondly, I guess people reading books regularly have more critical and like to make their own judgement. More than that, I think readers choose more wisely their books because it asks them an effort and it’s a real engagement where watching a movie take only 1 or 2 hours. For these two reasons I think that in the book sector the tail is longer, I mean that a larger part of the books sold doesn’t come from best sellers (in comparison with the movie sector).

Concerning the recommendations systems, I think they are useful and more present than before. Nowadays, people are more connected (by trivial relation sadly) and they are asking for the opinion of others. As an example, in the app-store or on Android, the apps are ranked by their quotation and the top ranked, the more recommended are the first you can see.

Now the question is to know if such recommendations systems impedes the long tail phenomenon or not… Taking the example of Netflix, I think they do. Indeed, I am more likely to recommend a movie or a series which is not very famous than the new blockbuster. For example, I would find it ridiculous to promote the 12 grammy’s series Breaking Bad, because I suppose every body knows it. I am more likely to recommend a new series for example. So from this supposition that recommendations systems promote unknown products, it surely impedes the long tail theory.
But can I say that it’s always true, probably not. It depends a lot of the type of recommendation system. On downloading websites, here is a ranking of the most downloaded products. If we accept that as an indirect recommendation system, we have to say that in some cases they reduces the long tail effect because it promotes the “best-sellers”.

In conclusion I would say that the long tail effect mainly depends of the type of products and of the way they arte purchased (on-line vs. off-line,…).

Sources:
-http://archive.wired.com/wired/archive/12.10/tail.html
-http://fr.wikipedia.org/wiki/Longue_tra%C3%AEne
-http://fr.wikipedia.org/wiki/Liste_des_distinctions_de_Breaking_Bad

The article is based on the theory of long tail. The theory was developed by Chris Anderson and describes the fact that the internet sites sell a high proportion of niche products in small quantities. This theory is often applied in e-commerce because the logistic costs are reduced and the centralization of products allows to manage products in small quantities.

Before, the economists referred to another law : the law of 20-80. The law of 20-80 led to place priority on customers that represent 20% of the total customers who contribute to over 80% of sales. With time this idea changed. Now the law of the long train is in place. With this law, the idea consists of paying attention to customers who represent 80% of the total mass and 20% of sales. Now, it is possible thanks to internet.

Instead of offering only the 20% of references that make 80% of sales, it is possible to be profitable by offering 80% of references that make traditionally 20% of sales. It is the revolution of internet, we can approach the niche markets. Firms sell small amounts of products but when you add the sales, they sell a lot. Amazon, the example of the article, makes 57% of sales with unknown references.
The reason being that the uncasual users want more than popular contents. The role of recommendations is to satisfy and retain these customers. For that, the site promotes niche contents of which the profit margin is more important than the popular ones and allow to diversify the offer.

After a few years, the results are not convincing. The long train effect is small and does not have any impact on the diversity. Today, we are witnessing the opposite phenomenon, some writers, musicians buy up all of demand. Now, we pay attention to « the superstar effect ».

According to Marcello Vena, responsible of numeric department RCS Mediagroup (books), the concentration of numeric market is the crux of the problem. The concentration of retail digital titles around a few global players makes the promotion of editorial diversity impossible. The monopoly is the problem. But Marcello Vena also says that the recommendations motors are truncated with their recommendations. To sell more they offer you another title of the same author than the original title of other books that you do not know. But it is the same problem with the social recommendations because the motor offers you the most recommendations shared and not recommendations with small views that could be an interesting discover. There is also a risk of media bias. Some recommendations systems could be associated with writers or musicians and generate a bias like we studied last week.

Nevertheless, I think that the recommendations are still important for e-commerce. For me, they replace the sellers in the physical shops. Indeed, in a physical store, the sellers help the customers to make their choices, but on the internet, there is no seller. Thus, the recommendation is used to help client for e-commerce.
So the recommendation system seems to act like the word of mouth. Indeed, we can consider the human beings like the best tool to communicate thanks to the interactivity and personal opinions. Before, the sites tried to have comments from authors or specialists in the field. But now, the sites try to attract recommendations from users and create a sort of community. This community votes and gives opinions. A book can be appreciated by the experts in the field and not be found interesting by the users. That is why, with time, the recommendations of users have emerged and become essential.
With the appearance of social networks such as Facebook, Twitter or GetBlue, the sense of community is reinforced. GetBlue allows users to indicate where they are but also recommendations for cultural goods (books, music,…). GetBlue gives notifications on what your friends do, what they listen or read and you can follow the activities of users with an option « friends to follow ». The friends give recommendations and the system allows to select its own sources of recommendations on the public profile of publications or tastes.

Finally, I think that everyone is sensitive to recommendations because these respond to our expectations. We want to be sure that the purchase that we will do is a good deal, we want to reassure ourselves that we made the right choice.

Sources :

• http://www.definitions-webmarketing.com/Definition-Longue-traine
• http://www.conseilsmarketing.com/autres-conseils-marketing/les-business-modeles-la-longue-traine-walkcast-plan-marketing-partie-65
• http://www.markentive.fr/blog/4-raisons-dutiliser-la-longue-traine-pour-votre-strategie-content-marketing/
• http://www.cairn.info/revue-d-economie-politique-2010-1-page-141.htm
• http://www.cairn.info/revue-reflets-et-perspectives-de-la-vie-economique-2008-2-page-95.htm
• http://lafeuille.blog.lemonde.fr/2014/06/24/pourquoi-la-longue-traine-ne-marche-pas/
• http://tech-insider.org/file-sharing/research/acrobat/0807.pdf
• http://www.longtail.com/about.html

I did not find any good readings proving or not what was only assumptions at the moment the article was written, but this comment will show some new ways marketers found to make us buy what we did not know whe needed.

Indeed, the article only spoke about « automatized » recommendations systems (by that I mean not taken care by an employee of the company website you are visiting), that directly appears while checking the page of a product. But same year as the publication on the blog was published, another one was posted by « Fortune » (link at the end), which stated that « Amazon also doles out recommendations to users via email. […] the company provides some staffers with numerous software tools to target customers based on purchasing and browsing behavior. But the actual targeting is done by the employees and not by machine. If an employee is tasked with promoting a movie to purchase like say, Captain America, they make sure customers who have viewed other comic book action films receive an email encouraging them to check out Captain America in the future. » I do not have information to assess if that sytem is still used nowadays, but I found it quite interesting to see that it was not only « computerized ».

Also, with the well known success story smartphone had, it seemed pretty obvious that it will have a rôle to play in the recommendation systems. In 2013, a project was being studied and tried, consisting on the implementation of an eye tracking system directly implemented in ordinary glasses, and connected to a smartphone in order to play the same role as an ordinary recommendation system, and also help the consummer in his decision process. According to the article i also linked in the end of my comment « instead of learning from click behaviour and past customer ratings, as it is the case in the e-commerce setting, the mobile RA (recommendation agent) learns from eye movements by participating online in every day decision processes. »
In a first pilot study they did with five randomly chosen participants in a supermarket, they tried to get a first impression of the user’s behavior in a situation where he would need such technology, and his acceptance of the technology. First results show long eye cascades and short fixations on many products in situations where users are uncertain and in need for support. They also found a surprising acceptance of the technology itself throughout all ages (23 – 61 years), but at the same time, consumers expressed serious fear of being manipulated by such a technology. For that reason, they strongly prefer the information to be provided by trusted third party.
Since the article do not give a name to that project, i wasn’t able to see the actual state of the project, but I think that some Google Glasses features may have been inspired by that kind of project.

– http://fortune.com/2012/07/30/amazons-recommendation-secret/
-http://pub.uni-bielefeld.de/luur/download?func=downloadFile&recordOId=2578942&fileOId=2602478

In the years following Anderson theory of the long tail the literature on the subject seemed to agree that the success of retailers such as Amazon.com and Netflix is largely attributed to the “ long tail” phenomenon. Indeed niche products, that are not available at limited-inventory competitors, generated a significant fraction of total revenue in aggregate. However most of existing recommender systems were not able to recommend tail products due to the data sparsity issue. Based on that realization interested parties then focused on developing effective algorithms to recommend long tail items.

Furthermore, in his 2010 book “ Music Recommandation and Discovery”, Oscar Celma assessed that Anderson’s success theory may have been a failure but that his core idea still is an interesting way to explain the web-provoked changes, in terms of the availability of all kind of products (from hits to niches). Later he states that a recommender should focus on promoting the tail of the curve by providing relevant, personalised novel recommendations to its users. It would imply smoothly interconnecting the head and mid regions with the tail, so the recommendations can drive interest from one to the other.

In 2014 more and more tend to think that this long tail theory might have been an utopia. Daniel Kaplan said “The long tail effect is so small that it can cause no change in the market and has no impact on cultural diversity”. Some even develop contradictory theories for instance the superstar effect where blockbusters sell more than ever. Thierry Crouzet went as far as saying : ‘We are witnessing the opposite phenomenon. The curve has certainly lengthened, but it has widened enormously. It has some titles that sell a lot, and all others who are fighting for crumbs.”

In her book “ Blockbusters” Anita Elberse also shows that even companies supporters of the long tail have changed their tune. On one hand, Amazon seeks above all to subcontract to other players, and is looking with Kindle to set foot in the editing and production. On the other hand, now that Netflix business model is migrating towards online video ( Netflix used to primarily send DVDs at home at the time of Anderson’s interview), the company is obliged to secure the license of his films where previously it only had to buy DVDs. However, these licenses become increasingly expensive. Netflix evolved into a studio and invests hundreds of millions dollars in the production and promotion of its own shows like “House of Cards” or “Orange is the new black” buying into the blockbusters theory. Youtube slowly seems to follow the same path.

So my conclusion would be that even if the theory of the long tail is valid it doesn’t pass the test of reality. They are economic factors that make its realisation impossible. Another thing is that we are constantly faced with too much information, too much choices on the web and it can be a source of anxiety. So bestsellerization and blockbusters are what work right now as they also answer the need for easy access to novelty.

SOURCES:

Celma, O. (2010). Music Recommendation and Discovery. Berlin Heidelberg : Springer-Verlag.
Chen, Y., Wu,C., Xie,M.,& Guo, X. (2011). Solving the Sparsity Problem in Recommender Systems Using Association Retrieval. Journal of Computer, 6(9), 1896-1902.
Yin, H., Cui, B., Li, J., Yao, J., & Chen, C. (2012). Challenging the Long Tail Recommendation. Proceedings of the VLDB Endowment, 5(9), 896-907.
http://lafeuille.blog.lemonde.fr/2014/06/24/pourquoi-la-longue-traine-ne-marche-pas/
http://www.slate.fr/tribune/84585/longue-traine-blockbusters