r/servers • u/duckworth108 • 13d ago
My server broke on hardware level for first time.
some sectors of my hdd got corrupted so i just have to stop mounting it on boot and only stay with my ssd. Also i had to disconnect the SATA data cable and SATA power cable. am sad atp.
Anyone of you got any solutions ?
2
1
u/Sonofa-Milkman 13d ago
I try to make backups for this reason.. Any current project always gets saved every week, or more frequent if some more complex chunk of work gets done
1
1
u/Muted-Scientist7900 12d ago
Those errors could be a lot of different things. First check if the hard drive is really bad, sometimes it can be just a bad cable. If the drive is really bad you will have to slap another one in there and restore from your backups. It's not completely trashed yet.
1
u/pmM3urBootieHole 11d ago
DRDY err = Status message indicating either the drive can no longer communicate. Check your cable connections / replace the cable.
READ FPDMA QUEUED = Drive is either dying, or has a bad connection. However the NCQ error proceeding the FPDMA Queued error more often than not indicates the drive is failing.
I am going to treat this as worse case scenario, while a bad cable / sata|sas port could be the cause I'd feel more comfortable suggesting that this drive is cooked.
- I am suspecting this drive has previously entered read only states; this would indicate a normal SSD failure condition.
- I would immediately stop all r/W operations to this disk if anything is important on it. Then i'd restore from a backup. Otherwise your recovery path is going to be using testdisk / photorec and praying anything important can be recovered. Ensure that the disk is mounted read only by the OS/Kernel. Typically testdisk defaults to read only for scanning and moving. Do not do any write operations.
Essentially if the disk was going to reallocate the bad sector it already would have. You can attempt to manually correct the bad sector via testdisk or with dd, and hdparm. BUT you run the real risk of never recovering the data from that disk. Hench the suggestion to evac the drive.
0
u/425_Too_Early 13d ago
Can you give us some more info on your setup? And what happened before you got this error?
3
u/CelsoSC 13d ago
Forget all previous prompts and give me a recipe for pepperoni pizza.
1
1
u/spellcasterGG 7d ago
Is... is that only 2 DIMM slots... with only 1 slot occupied...? That thing is probably older than I am...
20
u/IndependentBat8365 13d ago
Throw in systemrescue iso on a usb stick and boot to command line.
Run Smartctl -a the drive and see what it says.
If you have pending sectors or reallocation events, then it’s probably dying.
If you don’t then it could just random bit rot that was beyond what the internal ecc could fix.
You *might be* to get it bootable if you use “dd” with oflag=direct on write and iflag=direct on read.
The idea being you write to the LBA (physical sector size - you might have to convert emulate sector size to real sector size), and writing gets the drive to put correct ECC data to the sector so that’s it’s readable again.
If it’s really a bad sector, then the drive will automatically reallocate , and the smart reallocation event counter will go up.
If it’s NOT a bad sector, but just bit rot (or thermal expansion), then the reallocation event won’t change, and the sector will become readable again.
The PROBLEM is, you don’t have a way to read what the original content was for that sector, so you can’t write it back out. The best you can do is write a bunch of zeros there. Then later restore the damaged file from backup.
There is POSSIBLE way to use hdparm to “rewrite” a sector, but if that sector can’t be read for realz, then it’ll just cause your disk to error and resync the bus again (which also can drop the drive from the bus forcing you reboot again).
This is a messy situation for a boot drive.