Comments on: Podcast Alley Self Destructing?

By: Doug Kaye

Doug Kaye — Sun, 03 Apr 2005 19:25:13 +0000

Actually there are statistical methods you could adopt that would be much better based on the distribution of voting. You have three values: (a) the number of votes, (b) the average (mean) of those votes, and (c) the distribution of those votes. (a) and (c) determine the accuracy of (b). The most accurate rating is when you have a large number of votes all within a short range (the standard deviation) of the mean. As the number of votes drops *or* the distribution of the votes spreads, the accuracy is diminished. It’s possible to create a threshold at any level that takes into account all three variables, and below that threshold you have to say. “I don’t know because I don’t have enough data or the data varies too widely.” If you’re not a statistics junkie, you can still probably come up with some algorithms that works, so long as you’re willing to say “I don’t know” for perhaps the majority of sites you rank, because you probably *won’t* have a statistically significant for most podcasts. (The “long tail” is where comparisons fail, and that’s where most podcasts live.)

So why do Amazon ratings work so well? Because they’re not usually used to compare one book to another. A rating comes with a review, and it’s the review that’s truly valuable. The number of stars is merely an indication of the tone of the reviews, not a statistically significant number. Yes, you can search Amazon for highest-rated in a category, for example, but that’s far less accurate and merely a good way to get to the reviews of some of the books.

What I’d like to see isn’t a “Top ” site, but rather an Amazon-style site with good reviews and ratings. I don’t see the value, for example, in comparing Dawn and Drew with Reel Reviews or Coverville, but I’d like to know how each is rated by its listeners and why.

By: gary

gary — Fri, 01 Apr 2005 20:49:55 +0000

I am sorry I was not more specific. You must go to a podcast on the site to vote and see the voting method. If you login and go to a podcast you will get a better idea of the system. I will try and expain it. I will use casts number 1,2,3 in the top 25 section as examples. Cast number 1 has 18 votes with an average rating in the 4′s. The cast in 2nd place has more votes with 20, but has a lower average in the 3′s. Now cast number 3 has a rating of 5 higher than the 1st or 2nd place cast. It only had 5 votes, to cast number 1s 18 and cast number 2s 20 so they rank higher because of the number of votes is so much higher.

By: Doug Kaye

Doug Kaye — Fri, 01 Apr 2005 16:36:49 +0000

Gary, I went to your site but couldn’t find anything about voting.

By: Gary Leland

Gary Leland — Fri, 01 Apr 2005 04:17:58 +0000

Doug I have enjoyed your thoughts on voting and podcast directories. I would like to get an honest opion of our voting methods at http://www.podcastpickle.com

Thabk you

Gary

By: Kevin

Kevin — Mon, 21 Mar 2005 17:48:59 +0000

I’ve been a member of Podcast Alley for a month plus now and I’ve never once downloaded a show and listened to it based off of it’s ranking at the Alley. I look to see what others are saying about it first then listen to it. This goofy pissing contest over ranking is just that…goofy. While it’s nice to have some recognition, the fact is that the shows with larger listenership will always get better rankings than the ones that have lower listenership or even new casts that have a very, very small audience to start with.

I’m sorry but anyone with half a melon in their skull will realize that a shows ranking does not mean it is the best show out there. I’ve listened to the current #1 ranking show and while I don’t mind it, I don’t listen to it regularly since it is not my cup of tea.

Just my two cents – by the way, whatever happened to the cents sign on typewriters and keyboards? What a shame to have to fine a wingding for it instead of it being a key. Hell, ~ is on the keyboard and I’ve never used it. ~ should be replaced with the cents symbol

By: Dave

Dave — Mon, 21 Mar 2005 03:54:00 +0000

I’m new to podcasting and haven’t been able to get to Podcast Alley.

traceroute podcastalley.com
traceroute: unknown host podcastalley.com

By: Mike

Mike — Sun, 20 Mar 2005 03:31:54 +0000

Apparently it has self destructed. I haven’t been able to access the domain at all this evening.

By: philippe boucher

philippe boucher — Sat, 05 Mar 2005 23:37:21 +0000

Why be obsessed with this type of ranking?
of course it is better to limit to one vote since it is fairer but in any case I hope bloggers are a little more intelligent than relying on this type of “voting” to select the blogs they visit.
If not they get what they deserve.

By: Rob

Rob — Wed, 02 Mar 2005 11:42:27 +0000

Yes, I mean, the minute savvy listeners (who have heard a dozen or more shows) hear the current #1 ranked show, the credibility of the ranking and the site itself will go right out the window!

They may also want to consider a popular (i.e. “I enjoy it”) ranking AND a separate quality ranking. Honest listeners would not necessarily rank a show the same on both scales.

By: Craig Patchett

Craig Patchett — Wed, 02 Mar 2005 04:30:28 +0000

My suggestion: Clear the entire system of votes and limit everyone to one vote per show per person. The problem is that it’s hard to enforce the one vote limit since it’s easy to spoof IP and email addresses, so charge $1 per vote via Paypal with the proceeds either going to charity or to support Podcast Alley. Why not?!