Superb quality and amazing crafstmanship takes iog to the next level.
I am so sick and tired of sub-par iog.
I'm so sick and tired of sending people (you) links and not getting any credit for it.
Sorry, credit where it's due: Bphd told me about the rackmounted drive bay on newegg.
Okay, I so want this. Seriously.
I know, it's sick. But I can't help it. Pretty please? For Xmas? Anyone?
4 - I thought you weren't supposed to go cramming peripherals in random places just because you found the number of inputs inadequate?
If you're going the route of external enclosures, better to get one of those pre-fab gigabit SANs and be done with it, and USB availability and such be damned. The potential slowdowns (and bottlenecks if your home network is as convoluted as mine) just really don't matter unless you're doing an awful lot of munging on that data, and even if you are you'll have free "real" physical disk to spare. The cost isn't that bad either, depending on whether and in what flavor you want it to come pre-RAIDed, and how willing you are to abuse any potential academic/departmental/professional discounts that might apply; $250 for 500GB RAID1 is doable, if you're patient.
Aww, crap, my inadvertent po-facedness strikes again.
Soooo, in an attempt to educate myself about SANs (of which I had never heard before), I utterly failed. What's the story? Examples? I couldn't find anything that was priced and capabilitied remotely like what you described (having searched on "san") at newegg.
What does a prefab 500GB SAN get him that a 500GB internal RAID array wouldn't? You should do what you suggest, Ben, which is to get the larger drives and rebuild them in turn, and by the time you need more storage, more will be available for less, and you might be getting new hardware with more drive connectors, for all you know.
Ahh! In this as in most vaguely-Linux-related things, Wikipedia totally has your back: Storage Area Networks, which is to say those pre-fab hard-drive/RAID network appliance weidgets that have various sorts of logic for performing fileserver like functions. There's a whole range of complexity in terms of what disks they use, how big they are, expandability, and whatnot, but most of the "home" versions generally offer up their contents as SMBFS and/or NFS shares, which make them extraordinarily nice on heterogeneous networks, and eliminates the sort of availability problems you get when your "main fileserver" box has to be taken down to remake the world or whatnot. Granted, they're often not very secure, or the security options are limited and vendor-specific, but there are ways around it and the lack of hassle in being able to just plug one into your network and BAM! everyone has access to a new terrabyte is really, really awesome.
by the time you need more storage, more will be available for less
You know, I kind of said something like this, even though I have no idea what this post is about. But did he listen to me? Noooo.
Yeah, ogged, but some of those external solutions are, like, so much sexier.
I take it a new MB/case would be right out? I mean, you could probably (I don't know your setup) get the set for just as cheap as your multiple enclosure dealie. But you and Ogged are right. Just get two 500GB drives (Newegg has samsung SATA 500GB drives for $150 each). Of course, then the issue is that you lose the capacity of the two drives you already have...
And mostly a prefab SAN gets him not having to take the SATA enclosure and turn it into an approximation of a SAN, which might not be an issue if he only has the one computer. Plus, no connectors to speak of, assuming he gets his intertubes through some vaguely ethernet hub/switch-ish devise that has at least one spare RJ45 port.
what if you got new drives and cheap external enclosures for the drives you already have?
if he only has the one computer
I was assuming this was the case, but if it's not, and you want to share files across computers, a SAN is really nice.
I only use one computer at the moment, and am connected wirefully. If I got a laptop or something (say, if I won the lottery), I would probably just access the files on my desktop via NFS or whatever on it.
How much do you love me, Ben? B/c I might be able to sell a used powerbook soon.
Well, external loses all the fast about it. Disk is already a bottleneck, and adding network and/or bus to it isn't going to really help. But then, I have no idea what ben is doing with all of his disk.
But then, I have no idea what ben is doing with all of his disk.
Pleasuring your momAlmost exclusively storing, accessing, and modifying mp3s, oggs, flacs, and other audio media.
Breaking News: John Emerson's move to Denmark is immanent.
How long have you been away, td? Ben's been trying to shop his disk around to a willing partner. For sufficiently small values of "trying".
New MB/Case are inevitable like the dentist: the question is how much time does OP need until these are needed? It's basically a matter of capital.
Now that that's taken care of, back to the suggested topic: I've never been attracted to any animal in particular: is there something sexy I'm missing out on? I've never seen a dolphin, and they seem good candidates. Are they receptive?
Firewire is pretty fast, and even better, its comparative advantage is greater when you have multiple devices. Multiple enclosures and a firewire hub will work fine. Multiple enclosures and a USB hub may also work fine, but it might be a little slower.
Question, though. You seem pretty on the ball. Why not ditch this RAID 1 nonsense and have a nightly cron job to rsync your stuff from the internal drive to your external drive? Then you're protected against not only a drive crash, but also an accidental "oh shit I deleted all my files" or "oh shit I got some particularly virulent strain of spyware"?
Fair 'nuff. Though it's worth noting that some of the more retail-y SANs are effectively no bigger than a big USB/FW externals, and can be carted around to other locations with data intact, which can be pretty useful, kinda like a thumb drive on steroids. I'm carting one of mine back to my parents' house over the holidays, since I promised to help my dad digitize his record collection. Although I suppose the same is more or less true of a SATA enclosure, depending on your hassle-tolerance.
The accused, 20-year-old Bryan James Hathaway, also has a previous no-contest guilty plea of shooting a horse in order to have sex with it.
27: yes, it is. John Emerson's move to Denmark simply is the legalizing of sex with animals. For wherever a human and an animal come together, John Emerson is there.
I've never seen a dolphin, and they seem good candidates. Are they receptive?
You sir, are very new here.
Question, though. You seem pretty on the ball. Why not ditch this RAID 1 nonsense and have a nightly cron job to rsync your stuff from the internal drive to your external drive?
Because that presumes that I have a large enough capacity not only within, but also without—whereas this worry is prompted by the fear that I'm outgrowing that capacity. (Unless you mean: get one big drive and keep it inside, and another and keep it out. That's not a bad idea either; ISTR some technique of incremental backups that wouldn't even require a nightly copying-over of 200+ gigs.)
28: Everything you could possibly want to know about sex with dolphins.
That guy who got killed by having sex with a horse the hard way, the one that was captured on video now easily available on the internet? A documentary about the case is having its world premiere at the Sundance Festival.
Multiple enclosures, firewire, and software RAID.
Or you can get something like this: http://www.newegg.com/Product/Product.asp?Item=N82E16833201003
You also might get products more like what you're looking for if you search for "network attached storage" rather than SAN. But the people who have to store lots of data these days and don't need blazingly fast access in terms of I/Os per second are all moving to computers with local storage or super-simple external storage solutions.
Fuck rsync. Run your external CVS/perforce repository and version anything important.
(Unless you mean: get one big drive and keep it inside, and another and keep it out. That's not a bad idea either; ISTR some technique of incremental backups that wouldn't even require a nightly copying-over of 200+ gigs.)
That works, and is more or less what I do. I have a 100GB drive in my laptop and a 160GB external drive, that is split into two partitions. One of those is a copy of the drive in my laptop, and the other one is just for storage of... er... large video files.
If I wasn't such a cheapskate / used P2P software more, I could have bigger drives, or even one of those dual-disc external enclosures.
What OS are you running? Given that you pasted "/home" in the original post, rsync will do exactly the incremental backups you describe. Probably something like "rsync -a /home/ /external_drive"
38: including mp3s? rsync is fine, and can be easily cronified and forgotten about.
It's official: 98% of my storage space currently occupied by music. And it's all important, and I'm not going to version 30,000 mp3s.
Ben, it's time you started calculating the listening-hours left in your life.
25: A long time. I switched jobs, and the new job is still kicking my ass. I still lurk, but just don't have time to follow the comments all that closely anymore.
If it's all music, then by all means get a hub and a couple of external enclosures. MP3s are nice because they're big and generally get read from start to finish, and so do not tax the controller or the mechanism in any way, and rsync should happen with blazing speed, especially if they don't change much.
Rsync takes forever because I have lots of random source code so there are tens of thousands of 1k files, and the modification time has to be compared on each one.
What in tarnation is he stinking up the blog for, then?
Ben, it's time you started calculating the listening-hours left in your life.
I'm not going to stop buying books because I'll never read them all.
I was kidding. ;) I was going for hyperbole with the whole nerd arms war thing, like multiple backups in physically seperate and secure locations and encrypted filesystems on disk in case of theft.
46: It's part of the long-standing no one listens to bitch problem, I'm telling you.
Doesn't everyone use subversion instead of cvs these days anyway?
I'm not going to stop buying books because I'll never read them all.
One, books have resale value. Two, why, dear mother of God?
49: Would I sound totally sycophantic and dorky if I said that I listen to you? In as much as I went bphd -> unfogged and basically think you're completely awesome. Um.
50: In opensourceland, yeah, pretty much. I just never bothered upgrading on my personal out-in-colo-land server because of a certain amount of fatigue/laziness, and I use Perforce in house as a side effect of who I live with.
I'm assuming you're not a book collector; that, while you may admire books qua books, mostly you want to read them.
52, awww. It might be syncophantic and dorky, but I'm a total sucker for flattery. Thank you!
Everyone else: SEE?
Pretty much. Although the particularly hardcore seem to use perforce.
True, I am not a book collector; in fact, my mom sort of is, and I think it's silly to have so many first editions that she's never going to read—but these books she'll never read on principle, whereas the other books of hers that she'll never read she'll never read because life is polynomially bounded. (And of course she accumulates books at a truly fantastic rate, though she's started selling most of the miscellaneous crap publishers send her.)
It's late, and I'm veering off course. I'll probably end up with more books than I'll read. I can even see myself buying a book just because it's well-crafted, without regard to what it says. But none of this matters. And a digital music file isn't a sensual artifact and has no resale value. Why hoard it, if you'll never listen to it?
Do you live with a very large company, LR? With 5000 engineers or so? My new overlords use perforce, but I'm not sure we'll have to migrate our existing code over for a while yet.
You're welcome! (Also, not to sound like a creepy stalker, but gosh, PK is adorable.)
The Perforce thing seems more like an OS divide thing to me; at least, the people I know who use it do so because of it's integration with stuff like Windows context menus, and they're all professional developers who do proprietary/non-academic stuff, mostly on Windows. Do opensource people actually use it much? I feel like I have no indie nerd cred anymore.
"sensual artifact"? Good night, internet.
Dude, we are stipulating that you can't, because you would be dead. Stipulating, motherfucker.
Nope, but I live with people who work in a medium-sized-ish development studio that's owned by a large, borgish publishing house, and who are primarily Windows users and the type of people who are swayed by things like context menu integration. Who also occasionally a) work from home and b) have outside nerd-projects that ostensibly require versioning.
RE: backups: 100 Base T isn't the slowest rate in the world: you could write a batch file to archive all of your shit nightly to every computer you've ever owned.
a more difficult question: how can you tell if a dolphin loves you/wants you?
Yes, earlier links have covered this, but these tend to have been written by freaks,
Are there any other unusually sexy creatures?
if you'll never listen to it?
Not a given if you do random shuffle. The potential is there.
62: he is; the problem is, he knows it.
Shorter 64: Ben's a packrat.
~217GB of music is, if I'm guesstimating right, a little over four months of back-to-back listening. Which is totally doable, depending.
Are there any other unusually sexy creatures?
*whistles nonchalantly*
68: I kinda think most of the big cats are hott, in a way that is neither particular sexual or, like, furry.
And 71 was me.
Matt F has it right. We stipulate that I'll never listen to all of it, but there's still a nonzero chance that I'll listen to any one album, say.
70: all right-thinking people are. You all are probably those weirdos who believe in deleting email. 68: 100baseT is faster than a moderately-fast hard drive.
Actually, according to audacious, it's only 2519 hours—around 41 days.
I manage my packrat tendencies by not acquiring things in the first place.
As you say, Matt F has it right. I concede.
I don't believe in deleting email. I do believe in culling books and CDs. Am I sorry I parted ways with Catfish Rising those many years ago? No, I am not.
I'm probably going to delete this crappy 16 Horsepower album at some point.
I'm listening to a (so far) delightful (fairly) new album Wake:Sleep, by A Lily. w-lfs-n probably already has it.
Wow, that's weird. What I currently have in iTunes is ~110GB and it's supposed to be 66 days. Then again, that's mostly just hand-ripped stuff encoded at obscenely high rates. What does most of your stuff live as?
And as for listening to all of it, am I the only person who downloads lots of music specifically for reccomending/sharing with other people? Is that weird? Or maybe a sign that I need to stop taking eMusic's "people who liked X like Y" feature so damn seriously.
actually i'm betting 81 is gratuitious, and Ben has never listed to the album.
83, 84: I'm guessing FLAC files are responsible.
Or, more accurately, I can't remember how many hours are in a day. 104 days. Not even four months!
(There's a roughly even mix of mp3 and ogg, both at reasonably high rates—I calculated the ratio once, but forgot it.)
oh, I didn't think to check ben's math: I figured we all relied on computers for that stuff these days.
re: Firewire, you only need the one output. You ought to be able to chain the devices.
Isn't 100BaseTX for old, slow, people like me who haven't completed upgrading to a completely gigabit system? And doesn't speed depend on the throughput of the machine accepting the files? And what kind of fool lets his drives get 96% full, since 50% really ought to be the maximum?
Rebuild the array with new drives; save old drives as backup! Ya see. RAID is redundancy (because something could always go bad and write crap all over your virtual drive); drives sitting in the closet are backups. ALWAYS HAVE BACKUPS.
Also, a question: don't they have like chemical help for your problem these days, w-lfs-n?
m, seek professional
Okay, I so want this. Seriously.
I am now suffering psychological trauma.
I know, it's sick.
Extremely. I thought pimp cups were going to be codpieces.
But I can't help it. Pretty please? For Xmas? Anyone?
NO. But if you're going to engage in fits of masochism, I'll get you one of those instead since that seems appropriate punishment for your illness. Also, because it's fucking cold and I'm cranky.
m, nurse cratchet
78-81: I've been listening to very little besides Sparklehorse recently. The new album is really gorgeous.
Aw, man. An unfogged tech thread and I've missed most of the conversation. You people get up too early.
Well, responding to various things:
- 100 base T is definitely not faster than any reasonably modern hard drive. I think you're mixing up your b's and B's.
- You shouldn't use external drives as always-on storage, Ben, since very very few of them will spin down the drives when not in use. They'll die pretty fast if they're packed with consumer drives, which they will be.
- When did NAS start being called SAN? I'm not accusing anyone here of starting that trend, but it's been confusing me lately. It's storage that's network-attached, right? Not network that's storage-attached. Weird.
- Dan's Data has talked about a lot of this stuff. The site seems to be down at the moment (weird), but this article should get you started. I think he favors the Buffalo Terrastation. The RAID rebuild sounds like a good idea too, but would probably be a bigger pain in the ass.
- Never used Perforce. I think Subversion gets all the OSS geek cred these days (it's great).
- Finally, my own question. I haven't set up a RAID array before. I've got two 200 GB disks in my Linux machine and want to move everything over to them on a software RAID1 config, so that I can even boot off of them. I know this is possible (I've seen someone do it -- you end up booting off one of the disks with RAID off, then it turns on when you're in the OS), but I don't know how to migrate the OS and generally get this set up right. Is anyone aware of a good how-to? Or can anyone suggest a cheap hardware RAID card that works well with the Linux kernel circa Fedora Core 4?
"get up too early" s/b "stay up too late".
I thought that when it came to smallish reads/writes, the 6 MB/sec you can expect from 100baseT was actually more than an OK hard drive could be expected to do. Could be wrong tho.
And SAN (storage area network) used to be used to refer to the high-end fiber channel stuff where the storage looks like local discs, while NAS was usually used to refer to something like a NetApp filer that ran over NFS or CIFS, but I'd imagine that there's been the inevitable term creep.
I'm hijacking this thread w/my own computer question. It's embarrassingly simpler than all the stuff that's been discussed: I have a Dell Inspiron 700m laptop, about 3 years old. Lately, I haven't been able to run on plug power. Even when plugged in, it runs on battery. BTW, I checked and my battery isn't one of the recalled ones. Thoughts, anyone?
Try taking the battery out. I forget if there's a way to actually fix that kind of problem, though.
Does the battery charge?
Okay, I haven't tried that yet. Although I may have found my problem. I just plugged it in again and noticed small sparks coming out of the power cord, the part that goes from the black box thing to the computer. There's a couple of tiny holes there. We have a new kitten who's a chewer. Bad kitty! It seems to be running on plug power now. I'll try charging it up again tomorrow; I don't want to leave it running w/holes in the cord. Thanks and goodnight!
Yeah, that'll do it. You need to get a new cord, then.
If you're not careful, you'll need ot get a new kitten.
I was just at fry's and they had a terabit buffalo terrastation that could be configured as a 750 gig RAID 5, for like $500.
Ah, reminded me that I need another backup drive. Thanks, b-dub!
Better yet, with one of these and one of these, plus 4 750GB drives, you can have *3* terabytes for about $1200. (2.2TB if you go for RAID 5.)
I'm pretty sure the one you're talking about is the one I have, w-lfs-n, possibly even purchased from the same Fry's. I will vouch for it's awesometacularity.
You'd probably want some place to put that board, too.
And hell, I'll get in on the (re-)hijacking: w-lfs-n, don't you play the WoW? And, if so, how are you pulling that off under Linux/BSD?
w-lfs-n, don't you play the WoW?
'Fraid not, kiddo.
Crud; now I'm going to be cursing my senility and wondering who it was I had you confused with.
Lunar Rockette please email me.
I was just at fry's and they had a terabit buffalo terrastation
Two buffalo terrastations go 'round the outside, 'round the outside, 'round the outside...