Everything Else

I have contended for some time that Google has so much influence on the Web that it is altering it both with Adwords and with the Google search engine index. Some of this falls under unintended consequences and some of it is very much by design for Google's material gain. Google complains about SEO's gaming the search engines in a bid for higher rankings but I contend that while SEO's are gaming Google Google is gaming (read exploiting) the entire Web and that IMO this is not a positive thing for the Web.

Here are two excellent blog posts that support my point: How Google Killed Affiliate Marketing by Aaron Wall, and In Links We Trust - How Google Reshaped the Web by Digital Ghost. These are well worth the read.

Sources: V7N Search Marketing Blog and Threadwatch respectively.

Rich Skrenta and Valleywag both have good articles on how to beat Google in the search engine game. Those two articles combined are worth reading and I agree it can be done.

Of course the given in here is you have to deliver the goods in the form of good search results. Some requirements:

1. Relevant results: you cannot deliver crap. Relevence is decided by the public users . With that said you really do not need to beat Google on relevency only match them.

2. Your own index: using somebody elses search index won't do, it has to be your own index.

3. Fresh: The index needs to be regularly and constantly updated so as not to be stale.

4. Size: It has to be a big index and you have to be agressive about finding new pages to spider.

5. Agressive and deep spidering: You have to index dynamic url's and do so agressively.

But after doing that you can carve out a niche and do some proper marketing and you can probably take some significant market share away from Google.

For index and algo quality I think Yahoo could pull this off. 'Live' is not there yet and neither is Ask.

I am no fan of Globalization. I just do not see how exporting jobs to the Third World helps the people of the USA other than by lowering the price of my socks by a couple of bucks. The truth is we have been trained to always go for the lowest price retailer, which just plays into the whole cheap goods, Big Box Store mentality.

How About a Shopping Search for where goods are made?

I got to looking at shopping search engines like NexTag, Yahoo Shopping and Ask Shopping and most of these are set up to find the lowest price retailers. Why can't I search for, let's say, "socks" and restrict it to socks "Made in USA"? I just might buy American made socks.

I think I might like to try using such a search engine. If one exists please let me know.

I confess thaty I have not been paying attention to the blog search engines for the last couple of years, because I never use them. But recently I have been playing with them more and more. The ones I am looking at so far:

Technorati

IceRocket

Clusty Blog Search

Google Blog Search

Ask.com Blog Search

(I'm disappointed Yahoo discontinued their blog search, because I remember liking it at the time it launched.)

Are there any other blog searches I should be looking at? What do you use and why do you like it? Do you use several blog search engines and , if so, which ones?

New Search Engine for Aging Boomers. It appears that web search confuses and frustrates people over 50. So somebody has invented the aptly named: Cranky a search engine for aging boomers. Cranky is a part of a larger portal and social network for people 50+ called Eons.

The interesting part is that the core of Cranky is a directory of 5000 sites the Cranky editors have identified as being useful and of interest to age 50+ people. I'm not sure who is providing backfill results for when there is nothing in the directory to match a query but somebody has to, I cannot imagine a search engine being considered relevant for very long with only 5000 or even 500,000 sites indexed in this day and age. I can see giving pages from those 5,000 core sites preference in the SERP's.

To eliminate confusion, Cranky only gives 4 results on the first page of the SERP's. Personally I find this annoying, but this allegedly is to minimize overwhelming aging boomers with too much info.

From a business angle Cranky is a good idea: aiming a search site at a marketing demographic rather than a subject or geographic area since it will be easier to sell advertising to Madison Avenue traditional ad agencies that way.

The two parts that particularly interest me is 1.) the targeting of a marketing demographic (ie. aging baby boomers), with what is really a directory (here I was just saying directories are dead (silly me)) which I find exciting and a good idea for niche directories, and 2.) finding out which search engine is providing search backfill (un-human edited or ranked results) if any, for the Cranky search engine.

Source: Search Engine Land

So everyone is coming out with lists, predictions and report cards, at the risk of being a "me too" I'll post my own opinion of the Big Four search engines.
First, I divide search engines into two broad categories: 1. Stand Alone index, this means that the search engine's own index is big enough and the algo is good enough that you can use it alone; 2. Meta-Search quality, the index and algo are not quite good enough to use as your only search engine but are good enough to use as part of a meta-search engine.

Major Search Engines -

Google: Stand Alone. While I don't thing the index is nearly as fresh or as relevant as it once was, Google is still the standard by which all other search engines are judged. What I do like about Google is that they still tend to favor what they think are content sites over commercial sites if possible.

Yahoo: Stand Alone. Yahoo is the only other search engine that really can be used alone for searching. Yahoo seems to favor commercial sites slightly in ranking, which is good if you are shopping or searching for products, but can be annoying when searching for information. Still it is a quality index.

MSN/Live: Meta-search quality. MSN has one of the freshest indexes and most aggressive spiders in the search engine world. I like that they still give significant weight to on-page factors in ranking, and they are not totally dependent on link popularity schemes. Unfortunately, this leaves them open to spam. I think their quality is slowly improving and hopefully MSN will find a happy balance between on-page and off-page ranking factors. MSN is great to have in a meta-search engine because of the freshness and aggressive spidering.

Ask: Meta-search quality. I think Ask is getting close to Stand Alone status, but their index is still smallish, not as fresh as I'd like and their spider is much too timid to deeply spider sites, especially dynamic sites, so it is missing lots of pages of content. Still they have not got there in 2006 but I have hopes they can spider better in the near future.

Minor Search Engines -

Exalead - Meta-search quality. This French based engine is impressive and has a nice interface. Personally I like it a lot. However, on commercially competitive searches it still ranks too many spammy sites highly. Good as a secondary engine in a meta-search.

Gigablast - Meta-search quality. I think Gigablast should be included as a secondary engine in any self respecting meta-search. Gigablast will dig up and rank some good sites the majors overlook which makes it valuable.

Wisenut - Meta-Search quality. Wisenut has good relevancy, but not very fresh index. Good to have in a meta-search because, like Gigablast, it digs up and ranks some good sites the others miss.

Note: I use Ask, MSN, Gigablast and Wisenut indirectly every day for searching as part of Clusty meta-search, my default search engine. Those engines seem to do a good job for Clusty and other meta-search engines. When I need to dig deeper into a topic than a meta-search can provide, I use Yahoo and some directories like dmoz.org.

The scary part of this whole report card is that with the exception of relative newcomer Exalead, the report card is pretty much the same as it has been for the last few years. I think we might be stagnating badly in the search engine web index industry.

Jimmy Wales, the founder of Wikipedia announced that he is working on a new type of search engine sometimes called "Wikiasari" or Wikia Search.

It sounds to me that he hopes to somehow combine social networking and social editing like Wikipedia with a deep spidering search engine. Frankly, despite heckles from SEO's, it sounds like a noble goal.

Will it be a Google-killer?

Well the odds are against it, but that does not mean a worthwhile, usable and commercially self-sustaining major search engine cannot come out of this project. A lot of average Internet users, that I know personally use Wikipedia as a starting point in their searches so it is not unreasonable for Wikia Search to gain a following from traffic fed by Wikipedia.

I'm trying to ponder what Wikia Search will look like. A lot of people think Wikia Search will not scale, and that is true if you are rely only on humans to index the web. But if you also incorporate a unreviewed spidered index and a very good ranking algo then I think it can scale much better - if you keep that index fresh.

I'm a directory guy and while I recognize directories are fast becoming obsolete I have always thought that combining a human edited index of quality sites with a spidering engine on a large scale would be interesting proposition. Of course, Yahoo and NBCi used to do this years ago, but if you can actually spider the pages of the sites that have been human reviewed and broaden the participation in the human review process from just a few editors to a bigger base it might help. But it depends on how you do it: back in the old days when Searchking was trying to build a real search index it relied on human voting to determine relevancy, but since only a tiny percentage of users ever voted it rendered that ranking scheme almost useless. The lesson there is that most people just want to get find their information and leave in as few clicks as possible so very few are going to stop and vote of edit unless it directly benefits a basic need. That is something the Wikia people need to keep in mind because it almost invites spammers to game the system. Chris Sherman has more on social search.

I think the bottom line is, we still need some innovation in search engines, and we still need more major search engines with their own indexes than just the Big Four. Five major search engines would be better and six would be better still, especially if that will siphon off users from Google which controls too much of the search traffic right now. If Wikia Search can do it then I'm all for it.

Source: SEORoundtable

Oh, goody, I thought it was only me, but Danny Sullivan helps explain that most site owners see 70% or more of their search engine traffic coming from Google. That too has been my experience across most of my sites, although I have to admit as time goes on I actually do less and less optimization on all my sites, the numbers for Google traffic remain at about 70%+. Sigh.

In my own experience, traffic from general directories has almost disappeared. Years ago there was a time when Dmoz and Google directory delivered a lot of traffic in their own right. I still get some traffic from niche directories but I'm wondering if niche directories aren't also becoming just a sideline. Surprisingly I still get a goodly amount of direct click traffic from all those link pages that webmasters build on their sites and that most SEO's claim are worthless. Search engine's might discount those pages but people still find them useful. Finally I still get fair traffic to my various niche portals from StumbleUpon, so obviously surfing is not dead.

(Note: None of this applies to this blog or this domain (talmir.info) which I don't do any promoting or optimizing for and which I let spread strictly via WOM. That is pretty much on purpose just to see how traffic grows organically.)

I noticed a link on Michael's sidebar to the Snap search engine, which was enough to make me give it another look. The results I get on Snap are actually fairly good. Hmm this deserves watching. I see there is a way to submit a URL for inclusion but it is not clear if Snap is maintaining their own database or using one of their partners and are really a meta search engine.

One problem is that the site does not work all that well with the OmniWeb browser, which becomes a problem although it seems to be well behaved in browsers that share the Firefox rendering engine.

Anyway I'll try to test this over the coming months and give you some sort of impression.

*Snap.com (II) designates the current incarnation of Snap as a search engine. Originally Snap.com was a directory several years ago.

So my old rather disreputable "trucker" style cap finally fell apart due to advanced age, so I thought I would replace it with a search engine logo cap. I went looking for search engine gear to buy from the major search engines and was shocked to find that only Google has a functioning gear store! This gives Google a virtual monopoly on the gear front, not because they are better but because the other search engines have dropped the ball and IMO they ought to be ashamed of themselves.
Yahoo - first up on the shame list is Yahoo Gear Store. Jeez how lame is that. Forget peanut butter memos, this yahoo gear store has been closed since before June of 2005! Yahoo, if you want to compete with Google then you have to work on that attention span thing and show some pride.
Ask.com - Personally I like the Ask logo. So I was hoping to find a black ball cap with the Ask logo on it. Ask doesn't even have a store. Besides it looking cool I figured having "Ask" on my cap might be a good conversation flirting opener with chicks at parties when they ask me what they are supposed to ask me. See there is a whole domino effect here due to gear failure!
MSN/Live - I searched but found no store. Frankly, I was not keen on having a butterfly on a cap anyway.
Okay by now I was getting desperate...
Even venerable portals like Excite or Lycos had no gear, nor did newcomers Clusty, Exalead or Gigablast.
What about the directories? Dmoz.org was a bust. The little Mozilla mascot might have looked cool but no dice. So far only Uncoverthenet has gear, but no hats.
Man, all I wanted was to buy a cool looking search engine cap that didn't say Google - is that too much to ask? Damn. You can't compete if you quit the field.


Listening to ''I'll Be Home For Christmas'', by Frank Sinatra (Play Count: 14)

Check out Exalead search engine and their new look and new clean user interface. I'm going to start testing the search results too. My preliminary impression is very good.

Source: SearchGuild

Google jealously guards and remolds the linking structure of the web for it's own interests and profit. With Google being so dependent on links to determine ranking, would it not be in the interest of competitor search engines, who rely less on linking and more on on-page content to provide tools which would help corrupt that linking structure in order to bring Google down?

Just thinking.

The Paris-based World Association of Newspapers, whose members include dozens of national newspaper trade bodies, said it is exploring ways to "challenge the exploitation of content by search engines without fair compensation to copyright owners."

Full story at Editor and Publisher.

There have been rumbles and it was just a matter of time.

According to John Battelle they can identify your IP address - at least Google can. It makes you look at that 35 year cookie that Google has in a new light.

Is Google considering acquiring digital music subscription service Napster? That is the rumor reported by the NYPost. I think I heard a few weeks ago that Napster was starting to lay-off people.

Which would make sense for Google to buy a company like Napster, in order to get into the digital music business, rather than try to build it from scratch.

Jeremy Zawodny has issued an update on his experiment with selling sponsored links on his blog.

The search engines (one of which I work for) would rather that paid links be tagged with a rel="nofollow" attribute to indicate that any "link juice" or authority shouldn't be passed on to that site.

The problem here is I hate the way the search engines, particularly Google, control the debate on this issue. (Disclaimer: I sell all my advertising on all my sites through Adbrite which gives no link juice.) But webmasters ignore warnings from Google's Matt Cutts at their peril.

Here is my opinion:

Right or wrong has little to do with this issue, this is about Google being threatened by any artificial manipulation of linking. They can and will use their raw power to protect their turf. Webmasters may have the right to sell any kind of link they want but Google will probably do bad things to you if you do.

Google simply has too much market and mind share to be ignored. Ultimately they will fail to hold back the tide on artificial link inflation and their index will degrade (IMO it already has.) I also think that selling ads on a site per-click, like Adsense, does not favor the site owner because the advertisers still get branding value off the ad impressions even if there is no click and the webmaster is not getting paid for that by Google.

Marissa Mayer quotes.

Source: Jeremy Zawodny's Linkblog