Site Automation, Blocked IPs, etc

[Tim]

Making a new thread to move all this discussion, as opposed to the V9.5 bugs.

Quoting all the discussion so far, then deleting the posts from the other thread.

Aug 22, 2012 19:00:18 GMT -6 Felic said:

dont know where to put this but i am getting this page when i try to use the site

This site is currently unavailable


If you are the account holder, please contact customer service. Our customer service center is available 24x7 via phone, chat, or email:

Email: [e-mail]
Phone: [phone] (757-416-6575 Int'l)

Aug 22, 2012 20:00:03 GMT -6 Tim said:

I'm on it.

UPDATE: And by on it, apparently the 'it' is 'hold'. Been trying to get answers as to why the site is down for the last 30 minute.

Aug 22, 2012 20:44:19 GMT -6 Tim said:

Site back up.

Aug 23, 2012 6:42:28 GMT -6 King Richard said:

I'm still having trouble connecting to the site. :/

Aug 23, 2012 7:07:32 GMT -6 Tim said:

Ah crap.

Looks like the host blocked some IPs that were hitting the site the hardest yesterday). I'm guessing Stilgar (200k+ hits), KR (27k) and Baldur (25k) since you 3 were way above the next closest. I've sent an email to my host requesting that your IPs be unblocked. Sorry guys. I'll keep working on it.

KR & Baldur: Did you guys have some automation stuff running, or did you legitimately load over 25,000 pages from the site yesterday?

Aug 23, 2012 9:33:34 GMT -6 King Richard said:

I'm not smart enough for automation. And, I didn't even play yesterday. :/ I ran 123 turns off KR 36hours or so ago. But I don't think that math adds up.

Aug 23, 2012 10:16:10 GMT -6 King Richard said:

Also, I can't even get in with my phone, so that shouldn't be the same IP right?

Aug 23, 2012 13:37:04 GMT -6 Tim said:

I wouldn't think so. What error are you getting when you try to connect?

Aug 23, 2012 16:14:48 GMT -6 Baldur Nordal said:

I can't connect either. It looks like they are blocking my IP at a low level as I can't even ping the server.

I wasn't doing any automation from home - though I had multiple chat's open.

At work I was working on a quest scraper that I turned off (it's being blocked also).

-bn

[Tim]

UPDATE: After doing some more investigating of the info the host sent me, I think I may be incorrectly pinged Stilgar as the one with 200k+ hits. The IP address wasn't assigned to any characters, but it was doing a ton of wget commands similar to how his automation script works. I'm thinking now that this was Baldur's quest scraper. Stilgar's IP addresses were actually number 4 and 5 on the list, with 9529 and 9210 hits respectively. So still up there, but not the near the 200k.

Sorry for the false accusations, Stilgar! Didn't know anyone else had automation stuff they were working on.

I'm still gonna stick with no automation for now. Too easy for an automated script to go bad and start causing issues. I'd rather keep the site stable for everyone then accept the risk of this happening again. Sorry guys.

Still trying to get a hold of the host. Realized the email I replied to was for a specific support guy, so if he hasn't come back in to work yet it's not gonna get answered. Will try to contact their generic email. Sorry for the delay guys.

[Stilgar]

No problem about accusations, all I had to do is to stop the script.

9k more or less fits script + chat, though I counted less then 7k (and it's not guaranteed that I had 4 chats opened). Maybe I did 2-3k manually? Interesting that there's only two IP, while I have 4 a day.

Both KR and one of Baldur's IP were banned because of chats, and maybe something else (TAR?). So 8 people with that habit will give you another 200k.

[Tim]

All IPs should be unblocked now as of 5 hours ago. Let me know if anyone is still having issues.

Really, 2-3k manually probably isn't out of the question. If you have a couple hundred battles saved between multiple characters and you think about running battles, going back and forth between towns, selling loot, repairing gear, buying consumables, checking clan stuff, etc., it's totally do able.

I really don't think chat is a big problem. It refreshes twice a minute, which is only 2880 per day (not per hour like you had in your calculation I think).

I'm not sure what KR's deal was, but all the other hit use seems to make sense. I'll try to keep an eye on things though I don't see hits per IP, just total hits.

Sorry for the hassle guys. Hopefully this will be the end of this and we won't have anymore problems.

[Stilgar]

Aug 24, 2012 7:35:34 GMT -6 Tim said:

I really don't think chat is a big problem. It refreshes twice a minute, which is only 2880 per day (not per hour like you had in your calculation I think).

No, that's one chat, I had four. But I'm happy if load is OK now.

[Tim]

Aug 24, 2012 7:50:31 GMT -6 Stilgar said:

Aug 24, 2012 7:35:34 GMT -6 Tim said:

I really don't think chat is a big problem. It refreshes twice a minute, which is only 2880 per day (not per hour like you had in your calculation I think).

No, that's one chat, I had four. But I'm happy if load is OK now.

Ahh. You has 2880 * 6 hours * 4 windows. I saw it and though 2880 * 24 hours. My bad. Guess I should read posts better.