java-gaming.org blocked in Google?

Miscellaneous Topics
#1

Hi there,

java-gaming.org does not show up on any Google results recently.
Strange, just noticed that today.
Can anyone confirm that?

#2

I typed in java game loop and this site came up on the last entry of the first page

so it’s still there, just that google has given this site a lower priority I guess

#3

I tried many different names that would have been obvious to this site and JGO didn’t show up in any of the first 3 pages! O__O

#4

Probably not many links in from other sites, so low page rank.

#5

no, it used to be on second place, when searching for “java gaming” for example.
Dropped in ranking so suddenly since yesterday (without meaningful replacements)

looks like it was ranked down on purpose.

#6

I can find JGO thread on first page when I search solution for very complicated problems. Ofc it’s just my own thread which empty left unanswerable :slight_smile: (and looks like no one else has same problem like me over internet)

Riven may disable google crawler. FYI JGO has no google analytic.

#7

I checked it, and it seems the google crawler has changed. It used to hit the server at roughly 0.1Hz - 1Hz, but according to my logs, it sometimes peaked at 10Hz for over a minute, which my flood-control code (rightfully) deemed a Denial of Service attack. After 10 of these ‘attacks’, the IP was banned it for 3 days.

#8

Ah. Well, there we go. Why would Google crawl enough to be recognized as a DDoS?

#9

Well, the Bing crawler does it all the time. It killed my server twice, as there was so much swapping that I couldn’t login. The Bing crawler is pretty much banned every day.

#10

That’s a great way to ruin your ranking in the search engines. You can add a crawl-delay setting to the robots.txt and, if you register for Google’s Webmastertools, adjust the crawl frequency for the Googlebot in the settings.

#11

Well, obviously banning a crawler results in ruining the ranking. What else is new.

So far, google had this approach:

  1. measure time as T1
  2. fetch page
  3. measure time as T2
  4. wait for (T2-T1)*10
  5. go to step 1

This is what every sensible crawler should do. As by definition it will at most put a load of 10% on the server it is crawling.

Bing just blasts through, until the server is pretty much dead, and keeps on pounding the corpse. Therefore I’m not a fan of Bing, and couldn’t care less about having a high ranking on it. What is its market share anyway… 15% ? What is its marketing share among developers (or people that know how to change the default search provider in their browser) … 0.5% ? Screw 'em, and their abusive crawler.

#12

I agree that Bing isn’t really important, but if you can tame it by adding a simple entry to the robots.txt, it would be a win for everybody…so why not!?

#13

Because I wasn’t aware of it :point:

#14

Yeah screw Bing.

I was never too sure if the crawlers actually respect and adhere to whats written in the robots.txt :smiley:

#15

The ones that do aren’t the ones you have to worry about, like Googlebot.

#16

I am in Turkey and when I google for “game loops” this site is the third entry :slight_smile: