Recent site outage

Started Apr 29, 2012 | Discussions
Adrian Godong
dpreview AdminPosts: 400Gear list
Like?
Recent site outage
Apr 29, 2012

Hi all,

In case you're wondering, just now we recovered from an hour long outage on the site. Our primary DB server is dead and we have to fallback to our secondary. We do not expect any data loss to occur (it was a quick and painless death). The outage takes an hour because I faithfully tried resurrecting it (to no avail, apparently).

Sorry for all the inconvenience,

-- hide signature --

Adrian Godong
DPReview.com

 Adrian Godong's gear list:Adrian Godong's gear list
Nikon D60 Nikon AF-S DX Nikkor 55-200mm f/4-5.6G VR Nikon AF-S DX Nikkor 18-55mm f/3.5-5.6G VR +3 more
jpr2
Forum ProPosts: 13,474
Like?
Adrian: is the outage the reason for a wrong time-stamp...
In reply to Adrian Godong, Apr 29, 2012

...on newly posted messages (they are 60 minutes into future);
thanks for your valiant efforts on Sunday,

best,
jpr2
--
~
street candids (non-interactive):
http://www.flickriver.com/photos/qmusaget/sets/72157609618638319/
music and dance:
http://www.flickriver.com/photos/qmusaget/sets/72157600341265280/
B&W:
http://www.flickriver.com/photos/qmusaget/sets/72157623306407882/
wildlife & macro:
http://www.flickriver.com/photos/qmusaget/sets/72157600341377106/
interactive street:
http://www.flickriver.com/photos/qmusaget/sets/72157623181919323/

Comments and critique are always welcome!
~

Reply   Reply with quote   Complain
Doug J
Veteran MemberPosts: 7,549
Like?
Re: Recent site outage
In reply to Adrian Godong, Apr 29, 2012

Good recovery Adrian, thanks for the information.

Cheers,
Doug
--
http://www.pbase.com/dougj/birds_birds_birds
http://www.pbase.com/dougj

Reply   Reply with quote   Complain
Stephenhampshire
Contributing MemberPosts: 834Gear list
Like?
Re: Adrian: is the outage the reason for a wrong time-stamp...
In reply to jpr2, Apr 29, 2012

I quite enjoyed seeing messages posted "-37 mins" ago
--
Everything changes

 Stephenhampshire's gear list:Stephenhampshire's gear list
Samsung TL350 Pentax K-5 Pentax K-30 Pentax smc DA 16-45mm F4 ED AL Tamron SP AF 70-200mm F/2.8 Di LD (IF) MACRO +13 more
Reply   Reply with quote   Complain
reginalddwight
Senior MemberPosts: 2,003Gear list
Like?
Reason why we back up our data
In reply to Adrian Godong, Apr 29, 2012

.....for instances such as this.

Thank you for keeping us abreast.

Reply   Reply with quote   Complain
Jim Cockfield
Forum ProPosts: 14,312
Like?
Issues with post time stamps
In reply to Adrian Godong, Apr 29, 2012

New posts appear to have the wrong time info, as if they were made an hour ago, showing negative numbers (i.e., -60 minutes ago for a post you just made), making it impossible to edit posts you just made, even though the threads list shows the correct time (how long ago the post was made is incorrect when looking at a specific post)

My PC clock is correct (set to local time as EDT via Time Servers). The problem was not rectified by clearing cookies and logging in again. I've also rebooted and the problem persists.

This problem started this morning.

I just made a test post while typing this reply, and it showed it as -60 minutes old when I clicked on it to view it..

http://forums.dpreview.com/forums/read.asp?forum=1004&message=41366336
--
JimC

Reply   Reply with quote   Complain
Jim Cockfield
Forum ProPosts: 14,312
Like?
ditto for that reply (shows as -60 minutes old) (/nt)
In reply to Jim Cockfield, Apr 29, 2012
-- hide signature --

JimC

Reply   Reply with quote   Complain
Jim Cockfield
Forum ProPosts: 14,312
Like?
IOW, not adjusting for Daylight Savings Time
In reply to Jim Cockfield, Apr 29, 2012

IOW, it's treating my posts as if my time is set to Eastern Standard Time (GMT -5) versus adjusting for Daylight Savings Time, since my PC is set to EDT (Eastern Daylight Savings Time, which is GMT -4 right now), making my posts look as if they were posted an hour into the future with negative numbers for how old they are.

This problem was not occurring before your server issues this morning, and has not been resolved by clearing cookies, rebooting and logging into dpreview.com again, etc.

-- hide signature --

JimC

Reply   Reply with quote   Complain
Adrian Godong
dpreview AdminPosts: 400Gear list
Like?
Re: Adrian: is the outage the reason for a wrong time-stamp...
In reply to jpr2, Apr 29, 2012

I just checked the secondary server configuration and it is not set to the right timezone. I don't want to change this now because we'll have a proper rebuild on Monday. So for now we'll have to run with the one hour error.

Thanks and sorry,

-- hide signature --

Adrian Godong
DPReview.com

 Adrian Godong's gear list:Adrian Godong's gear list
Nikon D60 Nikon AF-S DX Nikkor 55-200mm f/4-5.6G VR Nikon AF-S DX Nikkor 18-55mm f/3.5-5.6G VR +3 more
Reply   Reply with quote   Complain
Hank3152
Forum ProPosts: 12,158Gear list
Like?
It also affected the edit feature.........nt
In reply to Adrian Godong, Apr 29, 2012

No text
--
Regards,
Hank

 Hank3152's gear list:Hank3152's gear list
Canon PowerShot S95 Canon EOS 5D Mark II Canon EOS-1D Mark IV Canon EF 300mm f/4.0L IS USM Canon EF 100mm f/2.8L Macro IS USM +5 more
Reply   Reply with quote   Complain
Jim Cockfield
Forum ProPosts: 14,312
Like?
Wait one hour and you can edit a post. ;-)
In reply to Hank3152, Apr 29, 2012

Tthe edit feature doesn't work with posts showing negative numbers for the post age in minutes (as if they were made in the future).

So, if you want to edit a post you make, just wait one hour (so that it shows it was just made for the post age) and you should have the usual 15 minute window to edit it.

That works. I tried it on the first post I made to this thread and it let me edit it after one hour had passed:

http://forums.dpreview.com/...forums/read.asp?forum=1004&message=41366336

Of course, that means that readers have probably already read it and may not realize it's been edited later unless they happen to read it again after an hour has passed and you've made your edits

-- hide signature --

JimC

Reply   Reply with quote   Complain
bikinchris
Forum ProPosts: 12,986Gear list
Like?
Well done
In reply to Adrian Godong, Apr 29, 2012

Thanks for backing up. It should remind ALL of us to back up your photos!
EVERY SINGLE HARD DRIVE EVER MADE WILL EVENTUALLY FAIL!!!!!

Back up your information people!

-- hide signature --

Good cyclists are:
Visible, Predictable, Alert, Assertive and Courteous

They also use the five layers of protection available.
Layer 1: Control your bike (Don't fall or collide with others)
Layer 2: Follow the rules (Don't be the cause of traffic crashes)
Layer 3: Use Lane position (Discourage other drivers mistakes)
Layer 4: Hazard Avoidance (Avoid other drivers mistakes and road hazards)
Layer 5: Utilize passive protection (Use protection when all else fails)

Chris, Broussard, LA

 bikinchris's gear list:bikinchris's gear list
Nikon D2H Nikon D3 Nikon AF-S Nikkor 17-35mm f/2.8D ED-IF Nikon AF-S Nikkor 28-70mm f/2.8 ED-IF Nikon AF-S Nikkor 200-400mm f/4G ED-IF VR +1 more
Reply   Reply with quote   Complain
JulesJ
Forum ProPosts: 30,644
Like?
Re: Adrian: is the outage the reason for a wrong time-stamp...
In reply to Adrian Godong, Apr 29, 2012

No probs, I replied tomorrow, you can read it yesterday!
Jules

Adrian Godong wrote:

I just checked the secondary server configuration and it is not set to the right timezone. I don't want to change this now because we'll have a proper rebuild on Monday. So for now we'll have to run with the one hour error.

Thanks and sorry,

-- hide signature --

Adrian Godong
DPReview.com

-- hide signature --

Julesarnia on twitter

Reply   Reply with quote   Complain
Guidenet
Forum ProPosts: 12,429Gear list
Like?
Adrian? A couple of questions please.
In reply to Adrian Godong, Apr 30, 2012

Are you telling me the Database Server has no parity, is not part of a RAID array with hot swappable drives? I mean Amazon owns DPReview and I'd think they could afford some protection of some sort other than just having a backup drive.

I'm also noticing that this backup drive's database doesn't have much active or is trivial in comparison. Is it a true backup or a hastily put together table to hold this data until you can rebuild the database tomorrow? I'm wondering if stats tonight will be reflected when the old database get's restored. For example. If an image in the gallery has 200 views and gets 5 more views tonight, what will it display tomorrow? I would hope 205 views but I'm betting everything is in status right now and it will read 200 tomorrow. I could live with that. The very worst would be the need to reset it all. I would hope that wouldn't happen. It looks to me right now views and other stats are not accumulating.

I've spent a lot of time adding keywords and whatnot. I'd hate to have to move it somewhere and do that all over again. That's not trivial.
--
Cheers, Craig

Follow me on Twitter @craighardingsr : Equipment in Profile

 Guidenet's gear list:Guidenet's gear list
Nikon D300 Nikon D700 Nikon D3S Nikon D800 Nikon AF-S DX Nikkor 18-135mm f/3.5-5.6G ED-IF +23 more
Reply   Reply with quote   Complain
svuori
Contributing MemberPosts: 615Gear list
Like?
Re: Recent site outage
In reply to Adrian Godong, Apr 30, 2012

Is it possible that notification emails weren't sent because of this? I have email notifications turned on for private messages but I didn't receive a notification yesterday even though I got messages.

BR,
Sampo

 svuori's gear list:svuori's gear list
Pentax K-5 Sony SLT-A77 Tamron SP AF 17-50mm F/2.8 XR Di II LD Aspherical (IF) Tamron AF 18-250mm F/3.5-6.3 Di II LD Aspherical (IF) Macro Tamron AF 28-75mm F/2.8 XR Di LD Aspherical (IF) +2 more
Reply   Reply with quote   Complain
Adrian Godong
dpreview AdminPosts: 400Gear list
Like?
Re: Recent site outage
In reply to svuori, May 1, 2012

Yes.

-- hide signature --

Adrian Godong
DPReview.com

 Adrian Godong's gear list:Adrian Godong's gear list
Nikon D60 Nikon AF-S DX Nikkor 55-200mm f/4-5.6G VR Nikon AF-S DX Nikkor 18-55mm f/3.5-5.6G VR +3 more
Reply   Reply with quote   Complain
Adrian Godong
dpreview AdminPosts: 400Gear list
Like?
Re: Adrian? A couple of questions please.
In reply to Guidenet, May 1, 2012

We're not expecting any data loss.

-- hide signature --

Adrian Godong
DPReview.com

 Adrian Godong's gear list:Adrian Godong's gear list
Nikon D60 Nikon AF-S DX Nikkor 55-200mm f/4-5.6G VR Nikon AF-S DX Nikkor 18-55mm f/3.5-5.6G VR +3 more
Reply   Reply with quote   Complain
Stephenhampshire
Contributing MemberPosts: 834Gear list
Like?
Re: Adrian: is the outage the reason for a wrong time-stamp...
In reply to Adrian Godong, May 1, 2012

Any update on when the datestamp issue will be resolved?
Thanks
--
Everything changes

 Stephenhampshire's gear list:Stephenhampshire's gear list
Samsung TL350 Pentax K-5 Pentax K-30 Pentax smc DA 16-45mm F4 ED AL Tamron SP AF 70-200mm F/2.8 Di LD (IF) MACRO +13 more
Reply   Reply with quote   Complain
Roland Karlsson
Forum ProPosts: 19,582Gear list
Like?
Re: BUT YOU CANT EDIT!!!
In reply to Adrian Godong, May 1, 2012

The faulty time stamp has made the editing function in practice non working.

Maybe you can wait an hour and then you can edit in 5 minutes - I dont know - I have not tested. And if that works - its not any help really

 Roland Karlsson's gear list:Roland Karlsson's gear list
Canon PowerShot G10 Pentax K-5 Pentax smc DA 15mm F4 ED AL Limited Pentax smc FA 50mm F1.4 Pentax smc DA 21mm F3.2 AL Limited +6 more
Reply   Reply with quote   Complain
Stephenhampshire
Contributing MemberPosts: 834Gear list
Like?
Re: BUT YOU CANT EDIT!!!
In reply to Roland Karlsson, May 1, 2012

I think it is fixed now
--
Everything changes

 Stephenhampshire's gear list:Stephenhampshire's gear list
Samsung TL350 Pentax K-5 Pentax K-30 Pentax smc DA 16-45mm F4 ED AL Tamron SP AF 70-200mm F/2.8 Di LD (IF) MACRO +13 more
Reply   Reply with quote   Complain
Keyboard shortcuts:
FForum MMy threads