Zugg Software :: View topic - Is the Google Search on the forum broken?

Beginner Joined: 10 Nov 2006 Posts: 22

Every time I try to search for anything I get a blank page of google results in a new window. It does not appear to be working correctly. Is there something I should be doing differently to make the search work? I know the answers I need are in the forum as I've seen them here before but google isn't finding anything! Evil or Very Mad

Enchanter Joined: 09 Sep 2007 Posts: 605

Try Forum Search
http://forums.zuggsoft.com/forums/search_kb.php

Posted: Mon Oct 29, 2007 7:53 pm

It does seem to be returning far too few results. I just searched for "trigger" and only had one result appear.

As Arde suggests, you can use the forum search as well. You can access it from the Site menu at the top of each page as well, if you don't want to bookmark that link.

Beginner Joined: 10 Nov 2006 Posts: 22

Thanks guys, this solved my problem. It appears the google search needs some serious tweaking.

Wizard Joined: 03 Mar 2001 Posts: 1127 Location: London

Agreed on Google needing some work - if you search for Lua then you get no hits...

Something else to note, if you use normal google search then it returns results
search for:
site:forums.zuggsoft.com lua

http://www.google.co.uk/search?hl=en&q=site%3Aforums.zuggsoft.com+lua&btnG=Google+Search&meta=

Wizard Joined: 03 Mar 2001 Posts: 1127 Location: London

Weird, seems to be one part of the search url that makes it return no results
If you remove the '&safe=active' part from the url generated by a zuggsoft google search then it seems to work much better

Posted: Wed Apr 23, 2008 7:05 pm

I'm not altogether sure Google is keeping anything it gets from this site for long enough. Take for instance what Guinn said two posts above, actually just click the link and you will see no results are returned.

The search could be so useful it really could but obviously only if Google actually grabs and keeps things. I have no idea how Google truly works since just think of the amount of webpages, the amount of words on the web in total, they surely don't have enough space to keep everything nor enough computing power to search it all instantaneously if they did.

So if this feature is ever going to work and give us something useful I think we need someone at Google to tell us what is going wrong!

Posted: Wed Apr 23, 2008 7:19 pm

Taz, I completely agree. I have been *very* frustrated with the Google searching. It *should* keep everything as long as it does in their main search engine. But there are topics that I can find via a normal Google search that do not appear in the customized search. So I also have no idea how it works. And getting any decent response from an actual person at Google these days is as hard as talking to someone at Microsoft or any other big company.

I used to be a huge Google fan, but I'm afraid that as they grow they are having the same support problems as any other big company.

But I currently plan to get rid of the Google search on this site and replace it with my own as soon as I have some time to work on the web site (in conjunction with some MyMuds stuff I hope). At this point I think my previous full-text search box was better than what we have now with Google.

It's possible that I'm doing something wrong with the customized search, but I couldn't find anything obvious the last time I messed with it, and the tools to actually determine how it is working (or not) are really poor.

Posted: Wed Apr 23, 2008 7:22 pm

Have you tried running through

https://www.google.com/webmasters/tools/sitestatus

and using the webmaster tools, perhaps that might get closer to an answer.

Posted: Wed Apr 23, 2008 8:40 pm

Yeah, I've done that. It all looks fine. No errors being reported. I didn't see any tool that let me enter a search term to just search the specific index (to test to see if certain pages are in the cache). So it's hard to tell if a specific forum post is still in the Google index for example. If you know of a way to test that, let me know.

Posted: Thu Apr 24, 2008 2:46 pm

The following picture could indicate a possible problem.

I can do more investigation with further tools but only if I can verify the site which may be something you don't want me doing, let me know by PM.

Posted: Thu Apr 24, 2008 4:55 pm

OK, that's definitely weird. Anyone know why that might be getting blocked? Is that something I'm doing on my site, or is it something Google is doing? Do you know what "line 5" is referring to?

Wizard Joined: 25 Mar 2003 Posts: 1113 Location: USA

Look at your robots.txt file(s)? Maybe it's referring to line 5 in there...

Posted: Thu Apr 24, 2008 5:42 pm

Ack, stupid Robot file!!!

There used to be a directory on our old server called /forum. So I had that old directory blocked. Apparently Google is assuming some sort of wildcard, so the line:

Disallow: /forum

also causes anything with /forums (with the s) to also be blocked. Geez that is annoying. Anyone know how to change the robots.txt so that "/forum" doesn't also match "/forums"?

Enchanter Joined: 05 Mar 2003 Posts: 593 Location: Canada

I have no way to test this but maybe instead of /forum try /forum/ with the terminal slash

Posted: Thu Apr 24, 2008 5:55 pm

OK, I think I have fixed it. I added a trailing / to all of the lines. I also found the diagnostic area in the web tools that shows the URLs that have been blocked by the robots file, and it was a huge list. So I definitely think this is a big part of the problem. It all happened when I moved from the phpbb directory to the "forums" directory.

Thanks for helping with this. Hopefully it will get better over the next week or so as the Google engine loads the new robots file.

Posted: Thu Apr 24, 2008 11:01 pm

You should look into the sitemap thing if you have time. Also you can up the rate at which the bot does a crawl, I think the sitemap submission may be part of that.