Mochabot log - CommonJS IRC channel: #commonjs on irc.freenode.net

2010-04-29:

[0:20] <tlrobinson_> kriskowal: have we heard any feedback from ES people about the recent binary proposals? last i heard they didn't like the earlier proposals
[0:29] <kriskowal> tlrobinson_ no word
[0:29] <kriskowal> they might just be pursuing the webgl proposal
[1:30] <Wes-> kriskowal: FWIW moz has already added typed arrays to spidermonkey mainline, primarily for WebGL
[1:31] <Wes-> kriskowal: ashb and I have been talking about building Binary/F out of typed arrays. We believe it is possible, including in the browser, except maybe for [[Get]] and [[Set]] -- will have to check catch-all support
[1:31] <Wes-> ('the browser' ~= firefox)
[1:34] <Dantman> You might be able to get away with an array of numbers for other browsers...
[6:47] <Dantman> Wes-, any tricks for spying on local traffic for debugging? tcpdump s
[6:47] <Dantman> isn't working out the way I want...
[6:48] <Dantman> I'm basically trying to figure out what the failing HTTP request a python script deep in a pile of libraries and dependencies I can't tweak is.
[6:49] <kuya_> you need it low level?
[6:50] <kuya_> firebug works...
[6:51] <Dantman> kuya_, "python" it's HTTP sent by a script.
[6:51] <kuya_> ahh not browser stuff
[6:51] <Dantman> ^_^ Nice I just pulled out my ethernet cord without losing my irc connection.
[6:52] <ondras> (because you were connected via wifi? :) )
[6:52] <kuya_> http://www.ethereal.com/
[6:52] <kuya_> is nicer than tcp dump :)
[6:52] <ondras> or wireshark
[6:52] <kuya_> yup
[6:53] <Dantman> ondras, wifi of course... but half the time I switch interfaces I expect connections to end up dying.
[8:12] <ashb> wireshark and ethereal are the same thing - just got renamed to wireshark
[8:20] <kuya_> oh
[12:14] <Wes-> dantman: make sure the traffic actually crosses a network wire. Use tcpdump or tshark to record the packets to disk. Use "strings" on the file to get a simple dump, or use wiresharks fancy packet following stuff if you think strings is making a mistake (you have to remember how strings works)
[15:07] <aristid> "A Buffer is a mutable, fixed-length, representation of a C unsigned char (byte) array."
[15:07] <aristid> this is terrible.
[15:09] <Wes--> aristid: respond with better text
[15:09] <aristid> Wes--: just drop the fixed-length.
[15:10] <aristid> it's not the first time that i complain about that limitation.
[15:11] <aristid> i really think Binary/B is a lot better than Binary/F
[15:11] <Wes--> I don't consider it big deal. I'll probably add a length property for my own use, FWIW
[15:12] <Wes--> But binary/F is better suite than Binary/B for implementing write-drain buffers than, IMO
[15:12] <Wes--> With Binary/B I wind up slicing the buffer quite a bit
[15:12] <Wes--> I would rather change the window (view)
[15:12] <Wes--> Although, Binary/B could do it cheaper if I shift instead of slice
[15:13] <Wes--> Assuming I optimize shifting in C to just move the pointer around
[15:13] <aristid> Wes--: you could also add a view class?
[15:13] <Wes--> sure, I could do that too
[15:13] <Wes--> six of one, half a dozen of the other IMO
[15:14] <aristid> Wes--: accumulating binary data in a buffer that grows as needed is too elegant to give up.
[15:14] <Wes--> aristid: So add .length :D
[15:14] <aristid> Wes--: well but i want to use it in the encodings spec
[15:14] <aristid> so i can't add it only privately
[15:14] <Wes--> OIC
[15:15] <Wes--> That might be something worth bringing up to the group if it's truly needed by encodings
[15:15] <Wes--> One important thing to remember, though, is that there optimizations to be had if the buffer never moves in memory
[15:16] <aristid> Wes--: maybe implement some sort of size-locking?
[15:17] <Wes--> Or write encodings to use Arrays of buffer :)
[15:18] <aristid> Wes--: yeah but then it costs a bit to convert it to a single buffer :)
[15:19] <ashb> if we are stuck with fixed-length buffers the encodings api comes a bit messier
[15:20] <ashb> (i.e. closer to iconv with lengths coming out, but we obviously can't pass in int*)
[15:20] <Wes--> ashb: the status object that I proposed for Binary/F solves that problem
[15:20] <aristid> Wes--: that doesn't sound un-messy :P
[15:20] <ashb> cliffnotes?
[15:21] <aristid> Wes--: i think the low-level encodings functionality should be separate but based on the binary API
[15:21] <aristid> and directly accessible.
[15:22] <aristid> if you add a state object to some binary API, then you destroy the beautiful separation
[15:22] <aristid> it also isn't clear to me whether it would be as easy to use as Encodings/A
[15:22] <ashb> aristid: tho having a mode that implements buffers as needed might work? more work on the implementation side
[15:22] <Wes--> encodings *must* need a state object, no?
[15:23] <Wes--> what happens if you only get part of a character?
[15:23] <Wes--> you can't block -- you've said it is async compatible
[15:23] <ashb> yeah the new Transcoder is the state
[15:23] <aristid> Wes--: Encodings/A keeps the state and all needed data in the Transcoder object.
[15:23] <ashb> (has buffers partial chars etc)
[15:24] <aristid> buffering partial chars is not really a problem because in sane encodings that's never a lot of data, and i don't care about trancoding performance for insane encodings.
[15:24] <Wes--> an ugly - but workable solution - is to keep a large buffer with the transcoder object, and select increasing views on, copying and enlarging as necessary
[15:24] <Wes--> basically, "realloc"
[15:24] <Wes--> on in JS space
[15:24] <Wes--> s/on/only/
[15:25] <ashb> aristid: what do we get if we remove the accumulator mode (and only accumlate when we have a partial char?)
[15:26] <aristid> ashb: and return single buffers for every call? possible, yes
[15:27] <ashb> (just thinking out loud)
[15:27] <aristid> then the user must deal with combining the list of buffers that he gets
[15:27] <aristid> in some cases he doesn't need to
[15:27] <ashb> yeah, if needed.
[15:27] <ashb> perhaps a Buffer (copy/concat?) that accepts multiple buffers to copy from
[15:28] <Wes--> ashb: var c = Buffer.concat(a,b)
[15:28] <Wes--> ashb: If you are very smart, you can cheat and turn a into c
[15:28] <ashb> Buffer.concat(a,b,c,d,e,f);
[15:28] <ashb> is the sort of thing i was thinking
[15:29] <Wes--> yep
[15:29] <ashb> also I didn't think /F had a concat?
[15:29] <Wes--> No, it doesn't IIRC, but if that's key to an elegant solution for encodings, then it should certainly be suggested
[15:30] <ashb> it might make it easier
[15:31] <ashb> certainly for cases where you aren't just transcoding the data and sending it straight back out
[15:33] <Wes--> I can also think of uses when assembling PDUs
[15:33] <ashb> not suer what PDU is in this context
[15:34] <Wes--> protocol data unit
[15:34] <Wes--> ashb: Do you think encodings/a can do GSM 7-bit compacted charset?
[15:35] <Wes--> i.e. 160 chars in 140 bytes?
[15:35] <ashb> should be possible
[15:35] <Wes--> that's about as stateful as it gets!
[15:35] <ashb> the one possible missing feature for encodings/a is how to add extra encodings in
[15:35] <Wes--> Since the value of every character depends on the one befor eit
[15:35] <Wes--> s/character/byte /
[15:36] <Wes--> and every byte depends on the byte after it, too
[15:36] <Wes--> wow, that's nasty to think about that way
[15:36] <Wes--> I just do the C bit-shift dance
[15:38] <ashb> some ay of doing require('encodings').register might ne needed in future
[18:58] <aristid> new! shiny! http://wiki.commonjs.org/wiki/Encodings/C
[18:59] <aristid> ashb: whatyathink?
[18:59] <ashb> i think i need a beer
[18:59] <ashb> (unrelated to that - not loaded it yet)
[19:01] <aristid> yeah get a beer and read the spec. what could be nicer than that?
[19:02] <ashb> the section on Stateful encodings says 'transcoder.close(output)' which is C+P from before i think
[19:02] <Wes--> Someone to read you the spec while someone else rubs your feet?
[19:03] <ashb> aristid: its certainly very minimal which is probably a good thing
[19:04] <aristid> ashb: yeah should be easy enough to implement.
[19:07] <aristid> i'm a bit worried about having to concatenate the resulting chunks but if you do it in one go, it should be relatively efficient.
[19:13] <aristid> Wes--: what do you think about Encodings/C?

 

 

Logs by date :