Mochabot log - CommonJS IRC channel: #commonjs on irc.freenode.net

2010-07-16:

[7:18] * Dantman wonders if he should add his HTTP-Gateway to the JSGI page
[7:19] * Dantman is trying to decommission the server commonjs moved off of
[7:56] <Dantman> I think I'll move my AIO ideas onto the wiki
[11:54] <Wes-> Dantman: If you're doing AIO design work, you should look at Event Completion Ports as implemented in Solaris 10 for architectural input
[11:56] <Dantman> Not really, I just started by moving notes onto the wiki
[12:41] <WesMac> zumbrunn: ping
[13:40] <dmachi> Wes-: I was curious as to whether or not you've implemented extended file attributes in your fs implementation?
[13:43] <WesMac> dmachi: No - I've never had a need frankly, so I didn't do anything past the traditional UNIX model. (No experience with them at all, unless you count permissions in Novell Netware 2.0.x back in the early/mid 90s!)
[13:43] <dmachi> k, seems to be consistently not implemented out of the spec :)
[13:44] <WesMac> dmachi: *nod*
[13:44] <WesMac> dmachi: also, I think there is a spec bug with default permissions umask
[13:44] <dmachi> ok
[13:44] <WesMac> I think 0777 should read 0666
[13:45] <WesMac> but it's been awhile since I looked at it
[13:46] <dmachi> we just had a filesystem store in pintura, and for some of those files it would be nice to keep some meta data like that. However, I figured it was a long shot as I dont' expect the lower level engines (eg the c based engines and such) to be in a rush to implement that cross-platform
[13:46] <dmachi> cross platform and cross filesystem for that matter.
[13:59] <WesMac> dmachi: GPSEE's target is really POSIX - but there are POSIX ACLs so I should probably look at better support someday.. But probably not 'till I need it! :)
[14:19] <deanlandolt> WesMac: do you have alternative ideas for a cross-platform place to tuck this kind of metadata into a file?
[14:22] <WesMac> deanlandolt: Not really. Best solution for metadata to go with files on a general case is probably extended attributes. BUT - most programmers will probably opt to reinvent the wheel and carry around some JSON anyhow
[14:22] <WesMac> which reminds me, actually
[14:22] <WesMac> we have an SQLite-based ORM module that carries around a database file + a JSON file to describe a complete entity
[14:22] <WesMac> using extended attributes, we could make that all one file
[14:23] <WesMac> but then I worry that I won't be able to shoot it from solaris to linux and untar it, y'know?
[14:23] <dmachi> yeah, thats the same issue
[14:23] <deanlandolt> yeah, we can certainly have a separate file but it'd be a lot better to avoid that
[14:25] <dmachi> deanlandolt: yeah i checked out the implementations and none support that at the moment. Its easy enough with the rhino implementations and more recent versions of java, but obviously much harder for the other engines.
[14:25] <WesMac> damn I wish POSIX would wade into this debate
[14:25] <deanlandolt> yeah, this would be really useful
[14:25] <WesMac> dmachi: I don't see why it would be harder anywhere else, the problem is portability, which IIUC java doesn't offer at the filesystem level anyhow
[14:26] <dmachi> i think the recent ones do have the implementations actually, thats why i meant it was easier :)
[14:27] <WesMac> so, if you copy one file with extended attributes from one OS to another, java guarantees that you'll be able to open it?
[14:27] <WesMac> If so, then we should steal their implementation
[14:28] <WesMac> but I don't understand how that could work
[14:28] <dmachi> not sure about that, what i mean is that it can read and set them on most of the fs's directly
[14:28] <WesMac> unless java insists that you use a java program to copy the file and transfer it
[14:28] <dmachi> whether you could just move them, i'm doubtful
[14:28] <dmachi> right
[14:28] <WesMac> dmachi: right, that's the really issue from my POV, although at least with java you only need to implement it once instead of ~6 times
[14:29] <WesMac> (calls are different between Darwin, BSD, Linux, Solaris, OS/2 and WIndows)
[14:29] <WesMac> Although, I only care about 4 of those
[14:29] <WesMac> The OS/2 bigots can do their own ports
[14:29] <dmachi> i'm not 100% the attributes wouldn't make it though, as many of the implementations seem to take care to make it work for things like nfs
[14:29] <WesMac> and the Windows people can run vmware :D
[14:30] <dmachi> WesMac: i think it gets multipled by platform and fs though, doesn't it?
[14:30] <WesMac> dmachi: nope
[14:30] <dmachi> maybe not, coiuldnt' tell for sure
[14:30] <deanlandolt> dmachi: w/o a guarantee that these xattrs get backed up using normal fs ops i don't think we could get away with that for the fs store
[14:30] <dmachi> deanlandolt: yeah definitely
[14:30] <WesMac> dmachi: the issue isn't so much "making it work for things like nfs" - it's making the extended attributes transferrable from box to box with normal tools like wget, ftp, sftp, tar, and so on
[14:31] <WesMac> deanlandolt: Right, that's the big problem I have with xattrs, too. I do /everythiung/ with tar-over-ssh, if tar doesn't support it properly at both ends with 100% fidelity, it's too risky for me
[14:32] <WesMac> and I'm not ever sure all those OSes can even store the same things
[14:32] <dmachi> http://en.wikipedia.org/wiki/Extended_file_attributes
[14:32] <dmachi> as a summary
[14:32] <deanlandolt> hmm...but are there /any/ other places to tuck arbitrary bytes?
[14:33] <dmachi> WesMac: what i meant by nfs, was that it seems like different os's have taken care to make things compatible for extended file attributes across nfs, which seems like if that ends up working it shouldnt' be out of th realm of possibility for tar
[14:34] <dmachi> but i'm mostly speaking out of my axe
[14:35] <WesMac> dmachi: That's a good point
[14:36] <dmachi> tar specifically has an option for xattr too but i dont' know if that means it does some normalization, presumably that tar code is itself dependent on the os/implementation
[14:37] <WesMac> that's GNU tar, but that's good enough (we install GNU tar everywhere, we like the -z option)
[14:37] * WesMac hmms
[14:38] <WesMac> I should look at that API in fs-base again
[14:38] <dmachi> i suppose we can just test it manually? add attributes on some windows box , tar and see what happens back and forth betwen some linux box
[14:47] <WesMac> dmachi: and leopard, bsd and solaris :)
[14:48] <dmachi> yeah, of course
[18:13] <zumbrunn> WesMac: belated pong

 

 

Logs by date :