Mochabot log - CommonJS IRC channel: #commonjs on irc.freenode.net

2010-03-30:

[0:36] <Wes-> kriskowal: ping
[0:36] <kriskowal> pong. not much time tho
[0:37] <Wes-> kriskowal: just wanted to let you know you - your(?) input need on list re. what constitutes dependencies on Async/A and how they are discovered
[0:37] <Wes-> kriskowal: monadic ensure strawman thread
[0:37] <kriskowal> ugh. i've been ignoring it.
[0:38] <kriskowal> i'm not really sure what the problem is.
[0:40] <Wes-> kriskowal: I don't understand how Async/A could possibly load deep dependencies of the modules it ensures without being told about them
[0:40] <kriskowal> Wes- when you've got require.ensure([x]), you should have a guarantee that require(x) will not throw. extrapolate the requirements therefrom
[0:41] <kriskowal> how it's told about them is orthogonal
[0:41] <kriskowal> and there are a variety of solutions on how to do that, none of which have an impact on the design
[0:42] <Wes-> kriskowal: So if require.ensure(x) has a module "x", and x has a method which does require("a") -- Async/A is responsible for loading a when ensure(x) is called?
[0:42] <kriskowal> what's important is that you make the promise to the programmer that their require calls will not throw
[0:42] <kriskowal> whoa. ensure has a module? i don't know what that means.
[0:42] <kriskowal> ensure has a callback and a list of module ids
[0:43] <Wes-> let me rephrase
[0:43] <kriskowal> and it guarantees that any of those ids can be required in the context of the callback. end of line
[0:43] <Wes-> this is module x: exports.A = require('a')
[0:43] <Wes-> I execute the statement require.ensure('x', cb)
[0:43] <Wes-> is 'a' loaded when cb is called?
[0:43] <kriskowal> ['x']
[0:43] <kriskowal> do you mean "loaded" or "required".
[0:44] <kriskowal> loaded means that the factory is memoized. required means the exports are memoized.
[0:44] <Wes-> kriskowal: bad choice of words
[0:45] <Wes-> function cb () { require('x').A; }
[0:45] <Wes-> is taht guaranteed to never throw with Async/A?
[0:46] <kriskowal> yes
[0:46] <kriskowal> Wes-^
[0:46] * Wes- scratches head
[0:46] <Wes-> How can that be implemented?
[0:46] <Wes-> without a synchronous require fallback
[0:47] <jbrantly> it seems to me that with require.ensure, "require" is still sync. So that while the specific modules defined by require.ensure are loaded, any deep dependencies arent necessarily loaded.
[0:47] <kriskowal> the transport format explicates the names of the modules that it requires
[0:47] <Wes-> AH
[0:47] <Wes-> So we *are* explicitly listing the dependencies /somewhere/
[0:47] <kriskowal> yes
[0:48] <kriskowal> shallowly or deeply, the server can opt to preload transitives
[0:48] <Wes-> The question is whether we list them with the monadic ensure, or whether they are listed in the transport system somehow
[0:48] <kriskowal> but it's orthogonal to the concern of how to request that guarantee
[0:48] <kriskowal> the transport. that's it.
[0:49] <Wes-> Except that you now have a requirement for a transport which knows how to discover the dependencies of a module -- unless we add some syntax that says modules are more than just their source code
[0:49] <kriskowal> there are numerous strategies for that.
[0:50] <Wes-> Oh, I'm not saying there aren't solutions -- I'm just trying to define the problem space more clearly
[0:50] <Wes-> Just saying "it will be so" in a spec doesn't make it happen -- we should be talking about the extra requirement on the loader in this case
[0:51] <kriskowal> ah, well, that's the scope of things. the transport is responsible for explicating dependencies, and the async call is responsible for making sure that the transitive dependencies, as listed by the transport, have arrived before the callback
[0:51] <Wes-> I don't think manybody else realize that Async/A puts a dependency resolution requirement on the module loader
[0:51] <kriskowal> do a close read of transport/d. you'll find that it provides all the guarantees any implementation of async/a would need
[0:53] <kriskowal> not to be rude, but i'm surprised it wasn't obvious that was the arrangement. the transport spec clearly sends the dependencies, which are obviously necessary for async/a to do its job
[0:53] <kriskowal> and there's also an opportunity for a dynamic loader to bundle more factories than were strictly requested, to reduce chattiness
[0:54] <Wes-> kriskowal: My assumption was that Async/A was orthogonal to transports - as such I didn't study those in details
[0:54] <Wes-> kriskowal: Basically, require.define() is extra meta data about a module
[0:54] <kriskowal> async/a is orthogonal given the requirements of transports as written.
[0:54] <kriskowal> yes, and a wrapper so it can be script-injected
[0:54] <Wes-> we're arguing semantics that I don't think need arguing
[0:55] <kriskowal> that's why i haven't been participating in the discussion, sorry
[0:55] <kriskowal> anyhow, i've gotta run
[0:55] <Wes-> The only thing I'm trying to get now - is that Async/A needs a Transports which resolves dependencies; Transport/D resolves them adding meta data to the module
[0:56] <kriskowal> all of the transports proposals are required to provide sufficient information about dependencies
[0:56] <kriskowal> some of the proposals are clear about what the minimal explications are.
[0:56] <kriskowal> for example, a define call that provides an internally consistent set of modules with no *external* dependencies doesn't need to explicate anything.
[0:58] <kriskowal> so you can have as little detail as no explicitly listed dependencies on the entire bundle, or you can have as much detail as the shallow dependencies and deep dependencies of every individual module, which would permit an loader to do much more surgical and computationally tedious loading
[0:58] <kriskowal> in any case, i've gotta be home 3 minutes ago. ttyl
[0:59] <jbrantly> Wes-: I don't get how Async/A needs Transports which resolves dependencies...
[1:00] <Wes-> jbrantly: Async/A says that require.ensure(['x'], cb) will make 'x' and all of its dependencies available to the callback cb
[1:04] <jbrantly> Wes-: Maybe I'm missing something with terms. A Transport is just a format, not running code (ie, it doesn't actually *do* anything like resolving dependencies)
[1:04] <Wes-> jbrantly: That's what I thought too
[1:05] <Wes-> jbrantly: But if you read Transport/D, they have added a require.define() section which requires you to list the dependencies of your modules
[1:05] <jbrantly> Wes-: Yes, so the wrapping code needs to be able to resolve, at the very least, shallow dependencies.
[1:05] <jbrantly> Wes-: By wrapping code I mean code that automagically wraps modules if you're not doing it by hand
[1:06] <Wes-> jbrantly: ...yes, and then get teh shallow depends of each of those depends and recurse
[1:07] <jbrantly> Wes-: Not necessarily. That work could be delegated to the loader (in this case, I mean the code running in the browser making the requests)
[1:08] <Wes-> jbrantly: my gut says you're right, but I'm having trouble with the details
[1:08] <jbrantly> Wes-: loader loads module A, which depends on module B. Before allowing module A to be require()d, it loads module B, and so on
[1:08] <Wes-> ah yes
[1:08] <Wes-> I was think of run time lazy loading
[1:09] <Wes-> Can THAT be done? I don't think it can
[1:09] <jbrantly> Wes-: Yes, I believe it can be.
[1:09] <Wes-> if module A is { exports.A = require("B") }
[1:09] <Wes-> if module B is { exports.B = require("C") }
[1:09] <jbrantly> Wes-: Remember that the Transports don't actually *run* the module code. The loader does.
[1:09] <Wes-> if module C is { exports.C = require("D") }
[1:09] <jbrantly> Wes-: So the loader decides when to run the module code.
[1:10] <Wes-> can D.js be lazily loaded at runtime when the JS programmer executes the statement alert(require"A")?
[1:10] <Wes-> I don't think it can
[1:10] <jbrantly> Wes-: no. It would have to be require.ensure(['A'], cb);
[1:11] <Wes-> jbrantly: Right - so Async/A may preclude lazy dependency resolution
[1:11] <Wes-> I guess, unless the module doesn't list the dep and instead of ensures it
[1:12] <Wes-> which is the monadic ensure pattern for that particular statement
[1:12] <jbrantly> Wes-: That last point is one I've been thinking about. In the case of a module ensuring a dependency, that dependency can be lazy loaded
[1:12] <jbrantly> Wes-: and, if possible, that dependency *should not* be included in the Transport unless its actually require()d outside of an ensure
[1:14] <jbrantly> Wes-: So, going back to your Module A,B,C,D. If ModuleA is required/ensured, then B,C,D also need to be loaded prior to the factory function for ModuleA running. So in that case, it's not "lazy loaded".
[1:15] <jbrantly> Wes-: But if ModuleC did a require.ensure for D, then theoretically D could be lazy loaded
[1:17] <Wes-> yep
[1:17] <Wes-> so we need the monadic pattern to get lazy loading and garbage collection
[1:17] <Wes-> but we don't have to mandate it in order to get correct behaviour
[1:18] <Wes-> provided that, instead, we use require.define() to define the requirements
[1:18] <jbrantly> righto
[1:18] <Wes-> oh, wait, you still need to mandate the monadic pattern to get garbage collectable modules
[1:18] <Wes-> but few people seem to care about that
[1:19] <jbrantly> the way I see it, require.define() only needs to list the shallow dependencies which are directly require()d. Anything else is optional.
[1:19] <Wes-> right, but the transport has to load them shallow and deep
[1:20] <jbrantly> No. The loader and/or the dynamic wrapping code can/has to.
[1:20] <Wes-> note that doing an internal ensure does not qualify as a dependency in this sense, it's orthogonal
[1:20] <Wes-> transport, loader...they have really be conflated in Transport/D if I understand it correctly
[1:22] <jbrantly> my understanding is that the loader is the code sitting on the browser (since we're talking about Transports here) which defines "require" and "require.define", etc. Transport is simply a way of wrapping code that is understood by the loader. As yet unnamed is code sitting on the server which takes a CommonJS module, figures out its shallow dependencies, and wraps it in a Transport format. Or you can do that last part by hand.
[1:22] <jbrantly> ... or through a build process
[1:24] <jbrantly> it seems to me that the unnamed part can either be smart or dumb. It can either figure out deep dependencies and bundle everything together, or it can be dumb and let the loader make a separate request for each dependency.
[1:28] <Wes-> jbrantly: right - my error in looking at monadic ensure was that the async module loaders & transports could allowably be as simple as "send the module file with no extra stuff, wrap a closure around it and eval it"
[1:29] <jbrantly> Wes-: I think thats possible, and I have it on my todo list to write it.
[1:29] <Wes-> jbrantly: but it has to be at least slightly more than that: somebody has to add a 'require.define' statement to the module which lists its dependencies
[1:29] <jbrantly> Wes-: *however*, that means that the code wrapping the module needs to figure out which dependencies are require()d by the module (I was just planning on using some regex)
[1:29] <jbrantly> Wes-: right
[1:30] <jbrantly> Wes-: but I don't see that as particularly hard. And it doesnt need to worry about deep dependencies.
[1:31] <Wes-> that was a big mental stumbling block for me -- which is one reason monadic ensure was calling -- monadic ensure makes the dependency problem "go away" by forcing the developer into a specific pattern of explicit declaration
[1:31] <jbrantly> so what you said, but instead of "wrap a closure around it", it's "wrap a Transport format around it"
[1:32] <jbrantly> Wes-: oh gotcha. Yea, I don't want to be forced into the require.ensure pattern. But I want it available in order to do lazy loading.
[1:32] <Wes-> jbrantly: No, it explicitly is not a transport format
[1:32] * Wes- nods
[1:32] <jbrantly> Wes-: why wouldn't it wrap into a transport format?
[1:32] <Wes-> I know people don't want to be forced into monadic ensure, which is why I didn't suggest it as a required feature, but more as one which needed to be explored to help tease out important details about the environment as a whole
[1:33] <Wes-> jbrantly: It COULD - but monadic ensure says /nothing/ about transport formats
[1:33] <Wes-> All monadic ensure wants is CommonJS modules written to use require() in a monadic ensure context. Full stop, that's the entirety of the requirement.
[1:34] <Wes-> Similarly, Async/A wants CommonJS modules with a require.define() statement
[1:34] <jbrantly> Wes-: oh, sorry. We might be talking around each other. I just mean if you load a module straight from the server as-is (not wrapped), then its easy/possible to take that module, wrap it in a transport format, and then eval it. It doesn't matter if the module was requested via require() or require.ensure()
[1:35] <Wes-> right, except that I suspect most people won't implement a non-ensure require, because that would be necessarily synchronous
[1:36] <jbrantly> synchronous in terms of "loading the entire dependency tree", but not in terms of "blocking the browser"
[1:37] <Wes-> jbrantly: require(X) must return module X's exports. You have to go synchronous to implement it without something like require.ensure()
[1:37] <jbrantly> Wes-: you're forgetting about the Transport. Remember, the *loader* determines when the module's factory function is called.
[1:38] <jbrantly> in other words, the synchronous require() statement is never even called until the dependency has been loaded. Thats the point of the Transport defining the dependencies.
[1:39] <Wes-> jbrantly: unless the loader has already retrieved it from the server, it can't run it on time: the module's exports *MUST* be available upon return from require()
[1:39] <Wes-> jbrantly: > It doesn't matter if the module was requested via require() or require.ensure()
[1:39] <jbrantly> Wes-: Right. But thats just it. The loader has already retrieved it from the server.
[1:39] <Wes-> jbrantly: I thought you meant just a require-out-of-the-blue
[1:40] <Wes-> jbrantly: like <html><head><script>require("X");</script></head></html>
[1:40] <jbrantly> Wes-: No. A require() inside a module. The starting point of the whole shibang needs a require.ensure.
[1:40] <jbrantly> *shebang
[1:40] <Wes-> jbrantly: right, we are in agreement, I misunderstood your statement
[1:40] <jbrantly> Wes-: or at least a "starting" module.
[1:46] <jbrantly> Wes-: I think this answers your question on the list, does it not?
[1:48] <Wes-> jbrantly: Yep - I think I understand the full ramifications of everything now
[1:48] <Wes-> real world differences --
[1:49] <Wes-> Async/A - more terse, cannot GC 'pure' modules, requires explicit dependency listing or machine-based dependency discovery
[1:51] <Wes-> monadic -- verbose, lazy loading is 'free', dependency discovery is 'free', can GC 'pure' modules (if pure modules self-identify)
[1:51] <Wes-> Real biggest difference is syntax, Async/A is much more user-friendly
[1:55] <tobie> tobie <3 irc log
[6:22] <Dantman> I should look at dynamic dns... I wonder if I could possibly hack up a dns server so I could run a dynamic dns client on my laptop that would update the ip on the hacked up dns, and the firewall would use.
[7:16] <Dantman> *sigh* I wonder if I'll have to pay for another year on that vps after all...
[10:01] <Wes-> Is there a way in Binary/anything to do a straight 8->16 bit string conversion?
[10:01] <Wes-> ashb: ^
[10:02] <ashb> dont thik so. but not sure exactly what you mean
[10:02] <Wes-> I recently found that binary/b's constructor is insufficient, even with charset = ascii
[10:03] <ashb> latin-1?
[10:03] <Wes-> ashb: I mean this:
[10:03] <Wes-> for (i=0; i < length; i++)
[10:03] <Wes-> buf[i] = ((unsigned char *)hnd->buffer)[i];
[10:03] <Wes-> str = JS_NewUCString(cx, buf, length);
[10:03] <Wes-> ashb: Hmm, does latin-1 map all 255 bytes straight across to utf8?
[10:04] <ashb> latin1 byte of 200 -> code point 200
[10:04] <ashb> i think
[10:04] <Wes-> *hmm* - still getting different bytes across the socket than I expect.. maybe telnet is also getting in the way?
[10:05] <Wes-> (I am playing with an echo server)
[10:05] <Wes-> ashb: btw did I tell you I wrote a reactor in JS? :)
[10:05] <ashb> no
[10:06] <Wes-> a SIMPLE reactor, but yes, it uses select() and node's tcp api
[10:09] <Wes-> and node's write semantic, which is sensible but a little surprising
[10:09] <Wes-> Have you played with node's tcp api?
[10:09] <ashb> no
[10:12] <Wes-> it's not bad. It requires a DOM-style event manager. Inoming connections and reads are events. Writes are non-blocking and immediate, but if the whole string doesn't write out at once, the remainder is queued up for the reactor to take care of when the socket is ready again; when the socket is drained by the reactor, a "drain" event is emitted
[16:56] <jbrantly1> ping Wes-
[17:26] <Wes-> jbrantly1: pong
[17:26] <Wes-> (quickyl)
[17:32] <jbrantly1> Wes-: I just asked in the list instead. No worries.
[18:59] <hammerdr> is anyone aware of any CommonJS code coverage modules?

 

 

Logs by date :