Mochabot log - CommonJS IRC channel: #commonjs on irc.freenode.net

2010-02-28:

[6:03] <Dantman> You know, now that I think about it HTTP had a potential way of making it possible for clients to send data without using a content-length (ie: In a situation where they don't know the length without buffering or precalculating something)
[6:09] <Dantman> A POST/PUT request could be sent without a Content-Length, contain a "Transfer-Encoding: chunked" header ^_^ perhaps even a Content-MD5 for validation... And if the server responds with "411 Length Required" then the client could retry the POST/PUT falling back to either buffering the data like it didn't want to or reading it twice to get the content length.
[8:16] <kriskowal> MisterN, how would one implement Transcoder.close() in terms of iconv
[9:00] <Dantman> *sigh* Code was looking fine... now for some reason Chrome seams to be posting stuff differently than Firefox in a way that breaks and makes uploads 0len.
[9:55] <Dantman> O_o Accept-Encoding: gzip,deflate,sdch
[9:58] <Dantman> Chrome is smoking something... quite surely... sdch, and sending a multipart message with the content-type and content-disposition but missing the actual content.
[10:32] <alfa> hi
[10:32] <alfa> i've got a string, without strange letter
[10:33] <alfa> and the browser says "unterminated string literal"
[10:33] <alfa> why?
[10:33] <alfa> that's my string
[10:33] <alfa> msg +='<ul><li><a href="?q=node/42" target="_blank" title="">1267199948 - News 4 </a></li><li>
[10:33] <alfa> News 4fa</li><li> </li></ul> '
[10:33] <alfa> +
[10:33] <alfa> '';
[10:34] <ondras> is there a newline+tab before "News 4fa" ?
[10:34] <alfa> newline
[10:34] <ondras> ah
[10:34] <ondras> that is your problem
[10:35] <alfa> is this?
[10:35] <ondras> you must end the string before newline
[10:35] <ondras> newlines must be escaped with \n if you insist on putting that newline into string
[10:36] <alfa> it's a generate string from php script, should not bw a newline
[10:37] <alfa> and if I use " instead of ' should I have the same problem?
[10:37] <Dantman> JSON encoder ftw!
[10:39] <ondras> alfa: yes
[10:39] <ondras> alfa: js strings cannot span multiple lines in source code, that is a rule
[10:40] <alfa> every day there something new to learn .... thanx a lot
[11:07] <ashb> ondras: not quite true
[11:07] <ashb> you can continue a string on a new line with a \ at the end
[11:23] * kriskowal grumbles
[11:23] <kriskowal> iconv() is segfaulting on me
[11:23] <ashb> :(
[11:24] <ashb> pastie your code and i can have a quick look if you want
[11:25] <kriskowal> oh. hm.
[11:25] <kriskowal> seems that my descriptor os 0xFFFFFFFF
[11:25] <ashb> that probab ly means cant go from charset to charset
[11:25] <kriskowal> which looks auspiciously similar to the invalid address i was getting out of gdb
[11:26] <ashb> 0xFFFF = -1 = error condition
[11:26] <ashb> if (p->conv == iconv_t(-1)) {
[11:26] <ashb> throw exception(....
[11:26] <kriskowal> yeh, i was doing < 0
[11:26] <ashb> that should'a given you a warning
[11:27] <kriskowal> that would have been nice?sec
[11:27] <ashb> i forget which class of gcc warning gives that
[11:27] <ashb> but its probably -Wall
[11:27] <ashb> and that much at least is useful
[11:29] <kriskowal> progress, thanks
[11:30] <kriskowal> ashb i'm implementing Transcoder from the encodings api for narwhal+node
[11:30] * Dantman really needs to get a credit card so he can get his own hosting, move his stuff there, and drop his old linode vps.
[11:30] <kriskowal> i now have opinions about it; i'll have to post something
[11:31] <ondras> ashb: yeah
[11:31] <kriskowal> yeah, money is handy. really helps with all that corporeal mucking around.
[11:32] <Dantman> Meh, Money isn't the issue... ;) Have plenty of that... ((And I really need to get that $100 cash out of my wallet...))
[11:34] <Dantman> :/ What's with everyone and requiring credit cards to do stuff online...
[11:35] <Dantman> I have 0 need for credit, 0 need for loans... The only thing I have use for is a method of paying online.
[11:35] <ashb> kriskowal: about encodings?
[11:35] <kriskowal> yeh
[11:35] <ashb> Dantman: how do you pay for it now?
[11:36] <Dantman> ashb, I borrow my father's credit card once every year (so far twice).
[11:36] <ashb> about all I can say for encodings is it works. can't remember much more than that
[11:36] <ashb> Dantman: ah
[11:36] <Dantman> And it's coming up again this year
[11:36] <ashb> you can't do paypal from a bank transfer in the US?
[11:36] <kriskowal> well, just sayin, no matter how bad set you are with the local government, i'm not wiring anything to you
[11:37] <Dantman> Problem is usually when you use a cloud host you usually do monthly.
[11:38] <kriskowal> ashb: yeah, encodings. i think the methods should be "transcode(source, target)", "reset(target)", "reset()", and "close()"
[11:38] <Dantman> ashb, Have a paypal connected to my Canadian bank account... though it's a week slow to add funds, and no decent host (dreamhost excluded) uses paypal.
[11:38] <ashb> Dantman: ah
[11:38] <kriskowal> i'll be in a better position to comment though once i've got this thing working
[11:38] <ashb> kriskowal: no OO you mean?
[11:39] <kriskowal> no, i mean those should be the methods of the transcoder
[11:39] <Dantman> For my iPod I use Mastercard Gift Cards.
[11:39] <kriskowal> definitely should be oo
[11:39] <kriskowal> to capture the iconv descriptor and the state
[11:39] <ashb> was gonna say :)
[11:39] <kriskowal> figured out how to dump the shift sequences
[11:40] <ashb> iconv(conv,0,0,...)
[11:40] <kriskowal> in any case, the proposal is mostly right
[11:40] <ashb> btw i've got some tests for the current behaviour if you want
[11:40] <kriskowal> might be handy
[11:40] <ashb> http://github.com/ruediger/flusspferd/blob/master/test/js/encodings.t.js
[11:41] <kriskowal> i'm using ryah's buffer instead of bytearray, but that's not a biggie
[11:41] <ashb> uses binary B too
[11:41] <ashb> shouldn't be hard to change
[11:41] <kriskowal> yeh
[11:42] <kriskowal> still haven't written the buffer(array) constructor though; i should do that.
[11:42] <ashb> its quite useful for cases like htis :)
[11:42] <kriskowal> what are the routines for getting a list of encodings?
[11:42] <ashb> :(
[11:43] <ashb> not portable is what
[11:43] <kriskowal> or normalizing an encoding
[11:43] <ashb> that's why it got removed from the spec - cos iconv doesn't do it
[11:43] <ashb> 'normalizing'?
[11:43] <ashb> as in unicode normalization?
[11:43] <kriskowal> canonical name of the charset
[11:44] <kriskowal> spec still says "list"
[11:44] <ashb> again - nothing portable
[11:44] <ashb> oh so it does
[11:44] <ashb> i thought that got removed
[11:44] <kriskowal> well, i'm not in love with it as is
[11:45] <kriskowal> but, again, i still don't have good criteria to judge.
[11:45] <ashb> ah so the OSX version of iconv.h has iconvlist and iconv_canonicalize functions
[11:45] <kriskowal> i've gotta use this a little
[11:45] <ashb> but istr that the glibc/linux version doesn't
[11:45] <kriskowal> grand
[11:45] <ashb> lemee check
[11:46] <ashb> yeah wow. it has iconv_open, iconv and iconv_close
[11:46] <ashb> nothing else
[11:47] <ashb> ah there might be a gconv_ fn
[11:47] <ashb> aha. there is a gconv.h lets see whats in there
[11:48] <ashb> hmmm nothing much.
[11:49] <kriskowal> ashb iconv can make progress *and* return -1 for an error, right?
[11:49] <kriskowal> it isn't transactional, i presume
[11:49] <ashb> iconv() you mean?
[11:49] <ashb> let me see what we do
[11:50] <ashb> kriskowal: not sure. we dont handle that possibility
[11:50] <ashb> oh yes we do
[11:51] <ashb> if errno is E2BIG or #INVAL
[11:51] <ashb> so yeah, it can return either of those and process some of the data
[11:53] <Dantman> *sigh* now to wait for dns to update
[11:58] <ashb> kriskowal: so a standalone libiconv should have a iconvlist
[11:59] <ashb> and the glibc version has http://bazaar.launchpad.net/%7Eubuntu-branches/ubuntu/karmic/glibc/karmic/annotate/head%3A/iconv/iconv_prog.c#L759
[11:59] <ashb> that horrible ness
[12:00] <kriskowal> wow, awesome
[12:00] <kriskowal> i like particularly that not all pairs work
[12:00] <ashb> yeah
[12:00] <ashb> its great :)
[12:00] <ashb> i'd suggest using ICU instead, but i'm not sure its better
[12:00] <ashb> and iconv is almost *everywhere*
[12:04] <kriskowal> well, my transcoder appears to work
[12:04] <kriskowal> (integrating with buffer toString et al)
[12:04] <ashb> cool
[12:13] <ashb> inimino: i think using numbers for hte node type is a false economy
[12:13] <ashb> cos in dealing with the parse tree you *cant* use the numbers: 1) it makes the code hard to read, 2) the numbers could easily change due to a small change in the gramar
[12:21] <Dantman> Anyone have Safari+QuickTime handy?
[12:22] <ashb> sure
[12:24] <Dantman> Show me what you see http://tests.danielfriesen.name/object-jpeg2000/
[12:24] <Dantman> ((If you see my old website then dns hasn't updated for you))
[12:25] <ashb> http://dump.ashberlin.co.uk/Screen shot 2010-02-28 at 12.28.33.png
[12:25] <ashb> http://dump.ashberlin.co.uk/Screen%20shot%202010-02-28%20at%2012.28.33.png
[12:25] <ashb> even
[12:25] <Dantman> T_T It works, but the quality is a little disappointing.
[12:26] <ashb> if i look at the images they show fine
[12:26] <Dantman> You mean the .jp2 is fine, but is broken in-browser?
[12:26] <ashb> no i mean its fine when i view it in browser
[12:27] <ashb> just not with the <object> shit you did :)
[12:27] <Dantman> Oh... T_T great
[12:27] <Dantman> That's disappointing.
[12:28] * Dantman wishes <img> tags worked like <object>, <video>, and <audio> all do.
[12:28] <ashb> object is not part of any standard
[12:28] <Dantman> No, it is...
[12:28] <Dantman> http://www.whatwg.org/specs/web-apps/current-work/multipage/the-iframe-element.html#the-object-element
[12:29] <ashb> okay so it might be in html5 then
[12:29] <Dantman> Just one of those things like innerHTML which was not standardized but widely agreed on, so html5 gave it an actual standard.
[12:30] <Dantman> s/agreed on/supported/
[12:42] <Dantman> ashb, http://tests.danielfriesen.name/object-jpeg2000/embed.html
[12:43] <ashb> only something in <object>
[12:43] <MisterN> kriskowal: are you still interested in the Transcoder.close() implementation?
[12:43] <ashb> it would help if oyu gve the img tag a src attribute, no?
[12:43] <kriskowal> MisterN i got it
[12:43] <kriskowal> thanks
[12:43] <MisterN> kriskowal: did you look at our implementation?
[12:44] <kriskowal> no, but it's pretty clear on close examination of the man page for iconv
[12:44] <kriskowal> the second and third forms
[12:44] <Dantman> Whoops
[12:44] <kriskowal> the catch is that those methods need to report their progress
[12:44] <MisterN> kriskowal: this is our implementation: http://github.com/ruediger/flusspferd/blob/master/libflusspferd/encodings.cpp#L200
[12:44] <kriskowal> close() needs to return the number of bytes advanced on the target buffer
[12:45] <kriskowal> i would like to call it reset() and reset(buffer)
[12:45] <Dantman> ashb, fixed
[12:45] <kriskowal> where reset() corresponds to just reinitializing the transcoder's initial state
[12:45] <ashb> Dantman: they all work now
[12:45] <Dantman> ashb, The <object> too?
[12:45] <ashb> yup
[12:45] <kriskowal> and reset(buffer) corresponds to reinitializing and writing the shift sequence to the given buffer, returning the number of bytes written
[12:46] <Dantman> Oh great... that's /nice/
[12:46] <MisterN> kriskowal: close does not return the number of bytes, it returns the binary...
[12:46] <kriskowal> m, and its length corresponds to the amount written, k
[12:46] <ashb> kriskowal: when would you call iconv_close?
[12:46] <kriskowal> close()
[12:47] <ashb> okay so keep close, just add a new reset method too
[12:47] <ashb> and make close just close?
[12:47] <kriskowal> that's my present idea
[12:47] <Dantman> So <object> works in Safari fine for jpeg2000... But not if you include fallback content... which is basically the soul purpose of using an <object> instead of an <img>
[12:47] <kriskowal> http://github.com/kriskowal/node/blob/narwhal-master/src/node_iconv.cc
[12:48] <kriskowal> i've only implemented Transcode(source, target), .transcode(source, target), and .close() so far
[12:48] <kriskowal> i'm not happy with my .transcode's return value
[12:49] <Dantman> <object> also triggers AdBlock in FF to go and add two separate block links... And using <embed> on a plain jpeg (nothing else) in FF causes a install plugins bar to show up even though it displays the image fine inside the embed.
[12:49] <kriskowal> i need to study the edge cases a bit more
[12:49] <ashb> given the buffer has .range, why doyou need _start, _stop ?
[12:49] <kriskowal> ryah will certainly want a way to bypass the extra buffer constructor
[12:49] <kriskowal> it's not free; he likes to have the option of doing things the hard way to make it faster
[12:50] <Dantman> Perhaps .range should be renamed then
[12:50] <ashb> then he can do it internally
[12:50] <ashb> It start to really complicate the API
[12:50] <Dantman> That extra buffer cost didn't exist in the first .range anyways.
[12:50] <kriskowal> optional args, easily ignored
[12:50] <ashb> there's the documentation penalty
[12:51] <kriskowal> oh, the extra buffer cost is negligible in comparison to the old .range's object constructor
[12:51] <Dantman> Eh?
[12:51] <ashb> if he's so worryed about speed he should want to avoid JS<->C transitions
[12:51] <kriskowal> frankly, i think it's all negligible and it's premature optimization since we haven't measured anything yet, but you can't compare what you haven't written.
[12:52] <Dantman> .range didn't have anything high cost besides the function call itself.
[12:52] <kriskowal> .range in node is cheaper than your proposal, dantman
[12:53] <kriskowal> it never leaves c++
[12:54] <kriskowal> no matter what you do with range, .range(0, 10).copy() is always going to be slower, albeit negligibly, than .copy(0, 10)
[12:54] <ashb> going JS->C->JS->C->JS is slower than just C->JS
[12:54] <ashb> so if he's *that* worred about speed he should not do things in js ;)
[12:54] <kriskowal> i'm not sure what your'e suggesting
[12:54] <Dantman> To be fair, that's .replace(b.range(0, 10), 0);
[12:55] <kriskowal> sure, i'm calling it copy
[12:55] <Dantman> .range(0, 10).??? is invalid in the old proposal.
[12:55] <kriskowal> i implemented it with memcpy and memmove depending on whether the regions overlap
[12:55] <ashb> kriskowal: just that if he wants that kind of interface, why doesn't he do it all in C before emitting it to JS
[12:55] <kriskowal> closures
[12:56] <kriskowal> by the old, you mean C. i mean D and E
[12:56] <Dantman> Heck, if you felt like it your .range() by C you could even return a string with an identifier in it.
[12:57] <kriskowal> i'm not sure what the fuss is about.
[12:58] <MisterN> kriskowal: i don't think our implementation has a lot of unnecessary stuff
[12:58] <kriskowal> flusspferd?
[12:58] <MisterN> kriskowal: you really should be careful not to break the transcoding. there are a lot of dangerous edge cases.
[12:58] <MisterN> yes, flusspferd.
[12:59] <kriskowal> in what context do you mean?
[12:59] <MisterN> i tried to be extra careful when designing the encoding API. yet the first few versions still had problems in some edge cases
[12:59] <kriskowal> i've got your test case.
[12:59] <MisterN> i'm not sure if our test cases are complete enough
[12:59] <kriskowal> i've got to adapt its asserts to commonjs
[12:59] <kriskowal> unit tests never are :P
[12:59] <MisterN> a lot of it was done by me thinking hard about what could happen
[13:00] <MisterN> (the API and the implementation, not the tests)
[13:00] <Dantman> ^_^ meh, I'm just saying that C's .range has less overhead than F's... The only overhead there is the cost that is implicit to the concept of .range being separate from the extra args in a .copy method.
[13:00] <MisterN> (the tests are written by ashb, not me)
[13:00] <kriskowal> yeah, it would be awesome if you could codify that for commonjs unit tests
[13:01] * Dantman wishes we had some sort of low-weight builtin host token object.
[13:01] <Dantman> Heck, setInterval could've used it.
[13:03] <kriskowal> MisterN, ashb, since we're all awake at this auspicious hour, i don't think you've chimed in on Binary/F
[13:03] <kriskowal> in general
[13:03] <Dantman> There goes nautilus
[13:03] <MisterN> kriskowal: i don't really understand the method descriptions
[13:04] <kriskowal> as of tonight, i've mostly massaged ryah's Buffer impl to Binary/F-ishness
[13:04] <kriskowal> the format of the method descriptions?
[13:04] <kriskowal> or the summary points for each
[13:04] * kriskowal pulls it up
[13:05] <MisterN> kriskowal: i'm still siding with Binary/B because the others don't seem to be that much better and we've already implemented /B
[13:05] <ashb> kriskowal: not sure i like write
[13:05] <kriskowal> so, with narwhal+node, i actually have all three?
[13:06] <kriskowal> i reused narwhal's generic binary/b impl, which uses a binary-engine module for the engine embedding adapters?
[13:06] <ashb> kriskowal: strictly speaking "ascii" charset shouldn't allow anything over codepoine 127 should it?
[13:06] <kriskowal> in any case, i wrote the node binary-engine using the node buffer type
[13:06] <kriskowal> aye
[13:06] <kriskowal> eascii gets you the full 255
[13:07] <ashb> do you mean "ascii" or binary/nop charset?
[13:07] <MisterN> eascii?? isn't that just latin-1?
[13:07] <kriskowal> conceptually, i can't speak for iana
[13:07] <kriskowal> i'm not sure.
[13:07] <kriskowal> eascii as i recall had block drawing characters
[13:08] <kriskowal> in any case, we're on the same page about ascii
[13:08] <kriskowal> where
[13:08] <ashb> wait - read is identical to toString isn't it ?
[13:08] <ashb> oh different error behaviour
[13:09] <kriskowal> different return value too
[13:09] <ashb> oh yes
[13:09] <kriskowal> i'm not terribly fond of read/write
[13:09] <ashb> no me either
[13:09] <ashb> i can see a vague kinda need, but i'm not sure i like them as they are
[13:10] <ashb> also is there much point in both copy and copyFrom?
[13:10] <kriskowal> i think it'd be better to stick with transcoding for real transcoding work, and keep toString(charset) on binary/f
[13:10] <kriskowal> perhaps even add Buffer.fromString
[13:11] <kriskowal> but just keep the subset of encoding that's safe for chunks in buffer, and leave the stream primitive in encodings
[13:11] <ashb> i think i'd prefer that
[13:11] <kriskowal> then we can build transcoding streams to make our lives easier at a higher level
[13:11] <kriskowal> which i think is the next thing on my todo list :-)
[13:12] <kriskowal> byte to text stream adapters, and vis versa
[13:12] <ashb> seekable text streams are *hard*
[13:12] <kriskowal> yeah. i'm thinking i won't do that.
[13:12] <ashb> :)
[13:12] <ashb> anything but go to begining or go to end anyway
[13:12] <ashb> tho even those have problems with a stateful encoding
[13:13] <kriskowal> yeah, that reset method would come in handy
[13:13] <kriskowal> the no-arg form
[13:13] <ashb> i wonder if some length properties (inputConsumed) etc might be helpful?
[13:14] <ashb> both in general and for the kind of uses ryah wants
[13:14] * kriskowal shrugs
[13:14] <kriskowal> i've gotta shut down; 5am
[13:14] <ashb> heh
[13:14] <ashb> g'night
[13:27] <Dantman> Time to murder this suicidal laptop... well, restart it that is.
[13:42] <Dantman> Great, things are still randomly crashing
[13:47] <MisterN> Dantman: is that os x?
[13:47] <Dantman> No
[13:52] <jammi> windows
[13:57] <Dantman> No, people should know by now I'm an Ubuntu user.
[14:16] <MisterN> Dantman: no i didn't know it
[14:18] <Dantman> *sigh* And it looks like there is a bug in Chrome... For some reason Chrome will omit the content of uploads it posts completely... You'll get the multipart message, the content type of the files, the content disposition with the name=, and filename= but the content of the fileitself will be completely blank causing you to upload a blank file.
[14:19] <Dantman> So now I have to reject blank uploads and warn the user that their browser could have a potential bug in it.
[14:20] <ashb> Dantman: thats *far* more likely to be a mistake you made
[14:20] <ashb> did you set the enctype of the form?
[14:20] <Dantman> Yup...
[14:21] <Dantman> It's completely fine... Uploads work fine in other browsers and all the right things are setup.
[14:22] <Dantman> ashb, http://pastie.org/846723
[14:23] <Dantman> Completely raw, nothing in between... a direct intercepted http post from chrome.
[14:23] <Dantman> It sounds like a bug in the linux version.
[14:24] <Dantman> There have been mentions of issues likely caused by it on the internet.
[14:28] * Dantman wonders what kind of Accept headers googlebot sends.
[15:13] <hannesw> Dantman: never had problems with uploads from chrome on linux
[16:33] <Dantman> Two things I would have liked in js... switch() like how Java does it... and for labelled break to work on {} code blocks instead of only on loops/switch.
[16:34] <Dantman> Though the latter one is partially a bit of confusion... I thought it worked at one point, had issues at another... so I have no clue if it's usable syntax or not.
[16:42] <ashb> what do you mean about switch?
[16:45] <Dantman> Different semantics.
[16:46] <Dantman> Firstly; switch("b") { case "a": print("a"); case "b": print("b"); case "c": print("c"); } prints "b", not "b" and "c"
[16:47] <Dantman> Multiple cases can still be combined.
[16:48] <Dantman> You use a {} block if it's more than just a single expression in the case.
[16:49] <ashb> deal.
[16:49] <ashb> its neither better or worse :P
[16:49] <Dantman> Implied break;... A huge gotcha related to switch.
[16:50] <Dantman> I mean, the way case falls through is a huge gotcha.
[16:52] <Dantman> ashb, If I remember correctly, something like this: http://pastie.org/846888
[16:55] <Dantman> Or maybe I'm thinking wrong
[16:55] <Dantman> ^_^ Lets just say something like that would be nice.
[16:56] <Dantman> Absent of the confusion and gotchas that switch/break have and Ruby's case/while.
[17:04] <Dantman> ashb, Maybe I was confused by Rhino's really clean use of switch; http://github.com/kriszyp/rhino-mirror/blob/master/src/org/mozilla/javascript/NativeString.java#L212-432
[17:08] <ashb> yeah very one of those has a return at the end
[17:08] <ashb> you are infact just making shit up :)
[17:08] <ashb> since if you look at line 170 it has break;
[17:09] <Dantman> Though I believe one of those has a continue in a way not supported by js... and it makes nice use of {}.
[17:10] <inimino> Dantman: labelled break can be used exactly as you describe
[17:10] <Dantman> Not break, continue.
[17:11] <inimino> ah
[17:11] <Dantman> Nonetheless I maintain that if cleaned up a little the idea has potential to be a much more readable and less unexpected version of switch .
[17:11] <inimino> well, that would be a major change
[17:11] <inimino> or goto :P
[17:12] <inimino> ashb: you definitely shouldn't use the numbers in code... but you can look up the indices once and then just compare on them
[17:12] <Dantman> Heh, cept continue is much nicer than goto.
[17:12] <ashb> inimino: what i'm doing right now is namelookup.RuleName
[17:12] <inimino> ashb: but most of the code I use that deals with trees is also generated so dealing with the numbers is easy...
[17:12] <ashb> and populating namelookup at file load time
[17:13] <inimino> ok
[17:13] <ashb> which is slightly less than ideal as its a prop lookup, but it guess its not too bad
[17:13] <inimino> ashb: but I'll probably add an option to use strings in the output instead of numbers for those who are dealing with the output directly
[17:14] <inimino> ashb: the difference in memory usage, at least in Firefox, is significant
[17:14] <inimino> but for small trees it's fine
[17:14] <ashb> my understanding was that if the string was the same then it should only use the memory once
[17:14] <ashb> but istr we had that discussion before
[17:15] <inimino> yeah, you would think so, but it doesn't actually seem to work that way
[17:16] <inimino> the other big problem was using the parser with Web Workers (which is my use case)
[17:16] <inimino> you are sending that back to the browser window from a worker, which means basically round-tripping through JSON
[17:17] <inimino> which takes a lot longer when the JSON is 5 times as long
[17:17] <ashb> yeah
[17:18] <ashb> http://github.com/ashb/pdoc-commonjs/blob/js-port/lib/pdoc/tree_walker.js
[17:18] <ashb> is my code that deals with the tree
[17:18] <ashb> brb - tea
[17:23] <ashb> inimino: any improvements you'd make?
[17:23] <ashb> inimino: i think i asked for this before, but a way to ignore certain nodes in the tree would be nice.
[17:23] <ashb> oh but that won't work with the current tree structure
[17:26] <inimino> ashb: ignoring leaf nodes can be done, I think that is in the stable version... you can pass an array of rules to drop entirely from the tree
[17:26] <ashb> oh yes
[17:27] <ashb> forgot about that
[17:27] <ashb> wonder if that helps me at all
[17:27] <inimino> ashb: but yeah, for things that have children it isn't as easy
[17:28] <ashb> EoF is the only rule i could safely drop
[17:28] <inimino> ok, yeah it depends on the grammar
[17:28] <ashb> yeah
[17:28] <inimino> in ES5 it was about half of them
[17:33] <inimino> well, not quite, but about half of the lexical grammar I guess
[18:34] <Dantman> Heh, now how do I test my Range: implementation.
[18:46] <Dantman> Oh great, I misread how If-Match works and have been working incorrectly as a result.
[19:31] * Dantman wonders if he should actually send base64 encoded sha-512's as ETags instead of hex encoded sha-512's.
[19:32] <MisterN> Dantman: base64 is cooler and saves precious bytes
[19:32] <MisterN> hex encoding reminds me of ugly uuids
[19:37] <Dantman> ^_^ base64 looks like garbage to me... I kinda like the simple HEX encoding,
[19:38] <Dantman> Oh well, Content-MD5 is base64 encoded anyways.
[19:39] <ashb> Dantman: what the ETag is doesn't actually matter
[19:39] <Dantman> I know that
[19:42] <Dantman> But we pre-calculate a md5 and sha512 when you upload the file, move the file to a shared tmp folder, and store the mime, size, md5, and sha512 in the database. Then when you save we use the sha512 to save the data data on the permanent filesystem like a hash->data map and use the mime, size, md5, and sha512 to create a new file entry in the database.
[19:43] <Dantman> So sha512 is the perfect unique identifier... we don't even ever have to calculate it after the first time you upload the file.
[19:46] <Dantman> Heck, we store a creation time and last modified time inside the database when you hit save.
[19:47] <Dantman> And when I implement updating a file that last modified will be updated, along with the hashes with the new upload data (and again hashes will be calculated souly on-upload).
[19:47] <ashb> md5 and sha512 seems excesive :)
[19:48] <Dantman> The md5 is so the Content-MD5 header can be sent.
[19:49] <Dantman> We use the sha512 only... but MD5 is calculated for Content-MD5 and perhaps anything that might still be using md5 instead of shas.
[19:49] <ashb> 'is an MD5 digest of the entity-body for the purpose of providing an end-to-end message integrity check'
[19:49] <ashb> is it really worth that?
[19:49] <ashb> i guess if the md5 and sha-512 algos are done right
[19:50] <ashb> its not much more expensive to calculate two at the same time vs just one
[19:50] <Dantman> Especially since the extra one is the md5 which is much faster to calculate.
[19:51] <ashb> still - what uses that header?
[19:52] <Dantman> Uploads are primarily images, but that's not the only things. Other than videos it's quite reasonable to expect someone will upload some software or something and offer it for download. Normally you hand out a md5 to ensure it was transfered right, not a sha512.
[19:52] <ashb> i mean do any UAs do anything with it?
[19:53] <Dantman> I suppose it could also be useful if we ever decide to use GridFS.
[19:57] <Dantman> Sounds like support for Content-MD5 in Firefox's download manager is on the todo list.
[19:58] <Dantman> I wouldn't doubt another download manager to support it too.
[20:02] <Dantman> Ah crap... I'm storing modified times down to the milliseconds, but HTTP only sends to the seconds, so I'm getting bad comparisons for last-modified.
[20:02] <ashb> :)
[20:05] <Dantman> *sigh* And now I realize the reason I didn't realize this earlier is because the filesystem also is only precise to the seconds so I got truncated times from there as well which matched up with the http dates...
[20:10] <Dantman> There we go... milliseconds truncated in all dates from a Last-Modified header, used to compare to a Last-Modifed header, or coming from FS.lastModified (in case there is a filesystem that actually does store milliseconds)
[20:11] <Dantman> Now back to coming up with a way to test if my Range implementation worked.
[20:24] <Dantman> ^_^ Looks like it's working after a fix now.

 

 

Logs by date :