Additionally, on Tuesday evening at our Mountain View ‘plex we're hosting the “Google Dance” -- where conference attendees can eat, drink, play, dance, and talk about search. During the Google Dance be sure to attend the “Meet the Engineers” event where you’ll be able to meet and have a conversation with 25 or more engineers including Webmaster Central’s own Amanda Camp. Also, if you get a spare minute from merry-making, head over to the Webmaster Tools booth, where you’ll find Maile Ohye offering lots of good advice.
If you’re a night owl, you’ll probably also be interested in the unofficial late-night SES after-parties that you only know about if you talk to the right person. To stem the potential barrage of “where’s the party” questions, I'd like to make it clear that I unfortunately am not the right person. But if you happen to be someone who’s organizing a late night party, please consider inviting me. ;)
"Enough about the parties -- what about the conference?," you ask. As you would expect, Google will be well-represented at the conference. Here is a sampling of the Search-related sessions at which Googlers will be speaking:
Meet the Crawlers Thursday, August 23 10:45am - 12:00pm Evan Roseman
We will also have a large presence in the conference expo hall where members of the Webmaster Central Team like Susan Moskwa and I will be present at the Webmaster Tools booth to answer questions, listen to your thoughts and generally be there to chat about all things webmaster related. Bergy and Wysz, two more of us who tackle tough questions in the Webmaster Help Groups, will be offering assistance at the Google booth (live and in person, not via discussion thread).
If you're reading this and thinking, "I should go and grab the last frozen juice bar in the freezer," I suggest that you save that frozen juice bar for when you return from the conference and find that your brain's overheating from employing all the strategies you've learned and networking with all the people you've met.
Joking aside, we are psyched about the conference and hope to see you there. Save a cold beverage for me!
Reporting is better, too. You'll now be told of multiple problems per line if they exist, unlike earlier versions which only reported the first problem encountered. And we've made other general improvements to analysis and validation.
Imagine that you're responsible for the domain www.example.com and you want search engines to index everything on your site, except for your /images folder. You also want to make sure your Sitemap gets noticed, so you save the following as your robots.txt file:
disalow images
user-agent: * Disallow:
sitemap: http://www.example.com/sitemap.xml
You visit Webmaster Central to test your site against the robots.txt analysis tool using these two test URLs:
http://www.example.com /archives
Earlier versions of the tool would have reported this:
The improved version tells you more about that robots.txt file:
We also want to make sure you've heard about the new unavailable_after meta tag announced by Dan Crow on the Official Google Blog a few weeks ago. This allows for a more dynamic relationship between your site and Googlebot. Just think, with www.example.com, any time you have a temporarily available news story or limited offer sale or promotion page, you can specify the exact date and time you want specific pages to stop being crawled and indexed.
Let's assume you're running a promotion that expires at the end of 2007. In the headers of page www.example.com/2007promotion.html, you would use the following:
The second exciting news: the new X-Robots-Tag directive, which adds Robots Exclusion Protocol (REP) META tag support for non-HTML pages! Finally, you can have the same control over your videos, spreadsheets, and other indexed file types. Using the example above, let's say your promotion page is in PDF format. For www.example.com/2007promotion.pdf, you would use the following:
X-Robots-Tag: unavailable_after: 31 Dec 2007 23:59:59 EST
Remember, REP meta tags can be useful for implementing noarchive, nosnippet, and now unavailable_after tags for page-level instruction, as opposed to robots.txt, which is controlled at the domain root. We get requests from bloggers and webmasters for these features, so enjoy. If you have other suggestions, keep them coming. Any questions? Please ask them in the Webmaster Help Group.
I have many different sites. Can I cross-link between them?
Before you begin cross-linking sites, consider the user's perspective and whether the crosslinks provide value. If the sites are related in business -- e.g., an auto manual site linking to an auto parts retail site, then it could make sense -- the links are organic and useful. Cross-linking between dozens or hundreds of sites, however, probably doesn't provide value, and I would not recommend it.
Greg: Like Shashi, this was also my first opportunity to speak at a conference as a Googler. It was refreshing to hear feedback from the people who use the software we work every day to perfect. The session also underscored the argument that we're just at the beginning of search and have a long way to go. I spoke on the subject of Web 2.0 technologies. It was clear that many people are intimidated by the challenges of building a Web 2.0 site with respect to search engines. We understand these concerns. You should expect see more feedback from us on this subject, both at conferences and through our blog.
Any special guidance for DHTML/AJAX/Flash documents?
It's important to make sure that content and navigation can be rendered/negotiated using only HTML. So long as the content and navigation are the same for search crawlers and end users, you're more than welcome to use advanced technologies such as Flash and/or Javascript to improve the user experience using a richer presentation. In "Best uses of Flash," we wrote in more detail about this, and are working on a post about AJAX technology.
