> We plan on having a separate i-mode crawl in which we'll use a
> DoCoMo-style user agent for spidering. (We already have separate crawls
> for WML and HDML, in which we use UP/Nokia user agents w/ a parenthesized
> "Googlebot..." at the end.) [Incidentally, will the parenthesized
> "Googlebot" throw anyone off? I'll find out soon enough, I suppose.]
Could you give us an example of the user agent you intend to use. With my
DoCoMo detection if Googlebot is in the device name section there would be
no problems with Googlebot being identified as an unknown new i-mode device.
> I like the robots.txt idea (User-agent: DoCoMo/*) a *lot*, but once again
> - is this something that most i-mode developers already do or would have
> to spearhead something?
I currently do not have a DoCoMo user agent in robots.txt as there is no
DoCoMo search bot but once we know the user agent of your i-mode bot, we
could add this to the file:
User-Agent: "Google i-mode search bot name here"
Disallow: "any non-i-mode content here"
This would both keep with the robots exclusion file specs and hopefully
solve any i-mode site identification issues.
Kyle
X-9 DESIGN LAB
http://www.X-9.com
[ Did you check the archives? http://www.appelsiini.net/keitai-l/ ]
Received on Fri Feb 16 03:31:10 2001