2010-11-01:
[16:30] <WesMac> deanlandolt: ping[16:31] <deanlandolt> WesMac: hey, still working up comments on your draft :)[16:31] <deanlandolt> sorry, been out of town for a wedding all weekend[16:31] <WesMac> deanlandolt: Awesome![16:31] <WesMac> deanlandolt: still working on draft. :D Wanted to ask you about 'protected' in java[16:31] <deanlandolt> sure, what's up?[16:31] <WesMac> deanlandolt: That's private-to-the-class data that can be shared among instances, right?[16:32] <deanlandolt> i should probably confirm w/ a java guru but yes[16:32] <WesMac> deanlandolt: What do you think about reserving the property module.protected for the module author? Then he can do friend classes / friend modules easier[16:33] <deanlandolt> ah, classes in the same package can access protected methods in java[16:33] <WesMac> (of course a plain "global" var would work too -- but module.protected could be "passed around" allowing other modules write access if so desired)[16:34] <deanlandolt> js doesn't really have a strong notion of packages like java so that could be a little confusing...but i think module.protected would be clear enough in its intent[16:34] <deanlandolt> but can we really ensure high integrity here[16:35] <WesMac> *nod* - thanks. I'm busy doc'ing the module namespace and want mostly to reserve a label for end-user-use[16:35] <deanlandolt> i've always been wary of pre-reserving but i can see the benefit[16:35] <deanlandolt> i mean, look at `protected`...reserved in js along with dozens of other keywords that have never seen use ;)[16:36] <WesMac> Yeah, pre-reserving has bit me before, actually, I think protected might have been one of the biters! :)[16:36] <WesMac> BTW IIRC ES-5 frees up some of those words, but we are still largely an ES-3 world[16:37] <deanlandolt> IIUC it frees them up to be used in a literal context, but not everywhere[16:37] <WesMac> Hm, that makes sense actually[16:37] <deanlandolt> yeah, there's no ambiguity[16:38] <deanlandolt> if only they'd done that from the beginning, JSON probably wouldn't require stupid quoted keys[16:38] <deanlandolt> (that was crock's primary rationale for quoting keys -- too many reserved word gotchas)[16:38] <WesMac> no, you still have to have them because of unicode and spaces, I think[16:39] <WesMac> OH[16:39] <WesMac> But he wanted &*everything* quoted,not just properties with special characters because of the reserved words[16:39] <deanlandolt> unicode, spaces and "-" and friends, yeah[16:39] <WesMac> because you can't determine with a simple regexp if a keyword needs escaping[16:40] <deanlandolt> yes, that was it -- the keywords complicated the regex[16:40] <WesMac> that makes total sense[16:40] <deanlandolt> i remember reading that and just WTF'ing...but i guess it does make sense[16:41] <deanlandolt> it just seems so crazy to bloat EVERYONE's serializations because of some silly regex bloat of ~200 bytes[16:41] <WesMac> OTOH it will make parsing faster[16:41] <WesMac> but, faster enough to justify the bandwidth? I have no clue[16:42] <deanlandolt> yeah, same...but it's the hand authoring, not the bandwidth, that really irks me[16:42] <deanlandolt> it's become /the/ transport format[16:42] <deanlandolt> (he probably didn't know that at the time, to be fair)[16:42] <WesMac> hee hee[16:42] <WesMac> I don't hand author JSON, ever, FWIW[16:43] <WesMac> I used to use Object.toSource(), now I use JSON.stringify() or whatever it's called[16:44] <deanlandolt> config files[16:44] <deanlandolt> ?[16:44] <WesMac> config files don't need to be JSON :)[16:44] <WesMac> check this out[16:45] * WesMac digs[16:45] <deanlandolt> i hand author every day when interacting with my data store backend...luckily kriszyp is a little more lenient w/ his parsing in json-ext[16:45] <deanlandolt> no, they definitely don't...but given json is THE format it's sometimes best (better than the OTHER format... *cough* xml *cough* )[16:46] <WesMac> deanlandolt: http://pastebin.mozilla.org/833405[16:46] <WesMac> Because it's a module and NOT JSON, I can even write some code to make hand-authoring the db schema even better than the db lib designer intended![16:47] <deanlandolt> WesMac: me likey[16:47] <deanlandolt> looks quite a bit like json-schema[16:47] <deanlandolt> (i think we talked about this once before)[16:47] <WesMac> And we can pull in project-relative db schemas like this:[16:47] <WesMac> require("db").connect(require("../etc/db_schema"))[16:48] <WesMac> or whatever[16:48] <WesMac> key is that the schema is now just another resource available to the application, nothing special[16:48] <deanlandolt> yeah, that's what we do in perstore, effectively[16:49] <deanlandolt> require("perstore/model").Model(require("resources/some_schema"))[16:49] <WesMac> awesome![16:50] <WesMac> This is the sort of thing why I really want to push module-relative resources as preferred over packaging solutions when doing so is reasonable[16:50] <WesMac> not that packages don't have their place -- but I prefer the subtle nuances of relative-require in many situations[16:50] <WesMac> and the logic/code is free, since we have to support it[16:51] <deanlandolt> agreed...the only place i think `packages` should come in is when declaring metadata (package.json)[16:51] <deanlandolt> there's some stuff you want to say about a body of code? great, say it with package.json and move on w/ your life[16:52] <deanlandolt> i wish git made it easier to compose packages a little better so you could subsume simple packages as modules, but git submodules blow[16:52] <WesMac> *nod* - although we DO need to solve the "how do we install packages and find their libraries" problem at some point in the not too distant future[16:53] <deanlandolt> that'd be nice...the /find their libraries/ part is totally up in the air[16:53] <WesMac> Installing something like your binary thing, or narwhal-lib, etc makes this a little more clear -- what's on the top-level module path, AND who should be able to write to it?[16:53] <WesMac> the latter question is crucial on a UNIX environment with many users using the same install of GPSEE in a root-only-writeable dir[16:53] <deanlandolt> have we solved the who should be able to write to it problem yet? does /2.0 address this?[16:54] <deanlandolt> can you attenuate the require?[16:55] <WesMac> No - it's sort of beyond the scope in a way.... but what we need is a way to do things like say "Hey! This dir in my home dir is a top-level dir" and "Hey! I'm running out of /usr/bin, but I want /X/Y/Z on my path, please"[16:55] <deanlandolt> the cap-based approach would probably be to provide something that can read from /X/Y/Z[16:56] <WesMac> I have considered comment-embedded require.paths but that's inelegant as it requires producing different executables for different values of X/Y/Z (which can vary from OS to OS)[16:56] <WesMac> deanlandolt: *nod* - actually, it might be possible to add that with fs-base and module.constructor.prototype from /2.0[16:56] <deanlandolt> yeah, this is something i haven't given *any* thought to...seems hard[16:56] <WesMac> gpsee sort of addresses this in a weak way[16:56] <deanlandolt> yeah, i was thinking perhaps you could provide a fn on module.constructor.proto[16:57] <WesMac> if you invoke with something *other* than /usr/bin/gsr -- i.e. /path/to/gsr, it will execute /path/to/.gsr_preload before invoking the script. This works from direct invocation and she-bang[16:58] <WesMac> So, we can produce project-specific interpreters, which link to /usr/bin/gsr, but give us an alternate require.paths[16:58] <deanlandolt> yeah, but hook conventions are tough to standardize on[16:58] <WesMac> extremely[16:58] <deanlandolt> still a nice idea :)[16:58] <WesMac> :)[16:58] <WesMac> It at least solves the problem I have at hand![17:00] <WesMac> we have multiple gpsee projects with conflicting namespaces and requirements, so we just give each project it's own gsr... then we tweak the environment before running the scripts and away we go, no need to edit the scripts just because we're moving to a different OS... just the preload file[17:47] <WesMac> deanlandolt: BTW, subtle observation, module.declare() can be implemented as this.declare = this.provide[17:47] <WesMac> recall that module.provide is very close in semantics to require.ensure[17:48] <deanlandolt> where was module.provide defined?[17:48] <WesMac> deanlandolt: probably somewhere around page 12 of the draft you have, under section 4[17:49] <deanlandolt> ah, gotcha[17:50] <WesMac> Getting module.provide into the spec means no more competition of require.async, require.ensure, etc -- and makes modules which need to lazy-load dependencies on the client still work on the server[17:50] <deanlandolt> yeah, i see[17:50] <deanlandolt> still trying to understand module.run[17:52] <WesMac> deanlandolt: I might still need to re-jig some of that stuff, as I didn't get the timing/semantics for when modules are initialized quite right in my first draft[17:52] <WesMac> it's important that modules are not required to be initialized until the first invocation of require[17:52] <WesMac> That requirement is important both from circular deps POV but also to maintain /1.0 semantics[17:53] <WesMac> or, more clearly, to make /1.0 semantics legal in /2.0[17:54] <WesMac> But, my goal is that by replacing various bits on the module.X namespace, a CommonJS programmer could monkey patch an arbitrary CommonJS environment to recognize new module formats, functionality, package systems, etc[17:56] <deanlandolt> that's a nice goal[17:56] <WesMac> Yeah. Tricky to get it to hang together in a way that's a) usable by the programmer but b) not restrictive to the environment author[17:58] <WesMac> I just learned, courtesty Merriam-Webster, that provide is a synonym for provide[17:59] <deanlandolt> oh? and i hear synonym is a synonym for synonym[18:01] <WesMac> and the antonym of antonym![18:02] <deanlandolt> the logic of your logic is unasailable[18:02] <deanlandolt> oh, and logical[18:03] <WesMac> Yeah, it's tight[18:03] <WesMac> In fact, you might say that it's taut-o-logical[18:04] <deanlandolt> heh![18:12] <WesMac> Let's see. If module.declare() is always invoked by module.provide(), and module.declare() returns the module factory function after module.providing() all the dependencies. Then we can hand off a closure to require.X which it can use to initialize the module and access it's exports. The initialization can be either immediate, or lazily (upon first reference of the exports)[18:12] <WesMac> Hm, probably cheaper to hand off require, exports, module, and the factory than the closure[18:33] <ls_n> Hey, this is Luke from the YUI team. I've been subscribed to the mailing list for a while, though delinquent in actually reading it regularly. Catching up on the digest this morning, there were some questions about YUI. I may be able to answer any questions about that, if there's someone on channel that was participating in that conversation.[18:35] <WesMac> ls_n: I was. I was interested in how YUI provides modules which can be directly loaded in script tags and/or how YUI modules are built with server-side tools[18:35] <WesMac> Also, how do you guys handle intermodule dependencies?[18:36] <ls_n> The module system is rooted in the YUI global object. so that script needs to be on the page first. After that, modules are defined via YUI.add('name', moduleFn, versionString, { requires: ['deps',...] });[18:37] <ls_n> So, if I'm reading correctly, similar to the wrapped module pattern you've been moving toward/settling on?[18:37] <WesMac> where do the deps come from? programmer-brain?[18:37] <WesMac> ls_n: Something like that - big different being that we infer name from URI and have no concept of versioning at the moment[18:38] <ls_n> yeah. No intrinsic dep calc. Seems to me that would require a decompile step (most likely in a build step)[18:38] <ls_n> there's no strict requirement for a build step in YUI.[18:38] <WesMac> And, so, what is the "build step" the poster intimated you use?[18:38] <WesMac> oh, okay[18:39] <WesMac> And your { requires: ['fdeps',...] } -- that's in the same file as the module source code, right?[18:39] <WesMac> So this is the "callee-specified" model I have been discussing[18:40] <ls_n> I usually start dev just hand coding YUI.add('blah', fn, ...) and go from there. We have a builder that uses ant. The build.properties file contains the deps. The module source is maintained without the YUI.add(...) wrapper. The builder adds the YUI.add(...) with the deps added in the config object that's the last arg.[18:41] <ls_n> Right, but we have a central collection of module deps, so we don't have to do serial module request->calc deps->fetch deps->GOTO 10[18:41] <WesMac> Okay, this all makes perfect sense[18:42] <ls_n> It's kind of awkward, and won't scale, so we're building server side solutions to do the full dep tree calc on a single request[18:42] <WesMac> We currently have three models of dep resolution, none of which are sufficient IMHO, and a wide-open ecosystem instead of a closed environment:[18:42] <WesMac> 1. "all dependencies magically satisfied" (server-side environment)[18:43] <WesMac> 2. Explicit dependency loading by module user (require.ensure)[18:43] <WesMac> 3. blocking XHR (is there a vomit emoticon?)[18:43] <ls_n> I'm interested in the areas where the YUI model is divergent from the direction(s) that are in play in CommonJS.[18:43] <ls_n> 3. we have something like that (serial async, not blocking XHR, though)[18:43] <ls_n> as a fallback for gallery modules[18:44] <ls_n> The YUI gallery is a community driven repo of modules, and we don't centralize the dep tree for those[18:44] <WesMac> Future direction, assuming it goes the way *I* want (a big if) -- is a dependency declaration at the top of each module which allows async dependency resolution and/or the trivial construction of server-side processes which deliver dependencies along with the module[18:44] <ls_n> but our module loader can fetch gallery modules without special instruction[18:45] <ls_n> WesMac: sounds familiar. I'm curious how this is diff than the YUI model[18:45] <WesMac> But, we have tough constraints, namely an open-ended module and the desire for a vanilla HTTP server to be able to deliver single modules without server-side processing or XHR[18:46] <ls_n> Isn't that just a hosted repo of wrapped modules, though?[18:46] <WesMac> ls_n: module.declare() ~ YUI.add() is part of the module, and the module name is defined by URI rather than the module's source code[18:47] <WesMac> ls_n: Not quite, one question is "where do wrapped modules come from"[18:47] <ls_n> I like the direction you guys are taking on the module IDs. The YUI model is pretty exclusive, which is unfortunate, but there wasn't really anything like it at the time.[18:48] <WesMac> *nod* - the module IDs are necessary to keep with the possibility of creating a secure module environment. You can't just believe what the module tells you. :)[18:48] <WesMac> It also pairs up well with the server-side environment[18:48] <ls_n> Where do wrapped modules come from? Seems like the 'module' ns/obj/what-have-you would have a path config of some sort.[18:48] <WesMac> (there should not be a functional difference between client and server if we do our job right)[18:49] <ls_n> right. We use our same module system for our NodeJS work[18:49] <WesMac> Right, is the "wrapping" of the modules that's the big question. If the wrapper is not part of the module format, then it requires a server-side or XHR step tocreate[18:50] <ls_n> aaaahhh, so "module" meaning module.declare('..', [...], function (...) { /* what's in here? */ });[18:50] <ls_n> == unwrapped?[18:50] <WesMac> my proposal for modules 2.0 has module source code looking like this:[18:51] <WesMac> module.declare([ deps ], function(require, exports, module) { /* Modules/1.1 code */ })[18:52] <WesMac> and, for example, you could probably retrofit NodeJS to add Modules/2.0 capability by running this during the bootstrap: module.constructor.prototype.declare = function(d, f) { f() }[18:55] <ls_n> Yeah, as you can imagine, I'm completely on board with the notion of the wrapping being a part of the module spec (because I've seen it's benefits in my day to day job).[18:55] <ls_n> I think the inlining of the deps in the array is nice for simplicity and readability, but in our case, the config object that includes the deps in the YUI.add() sig supports further configuration or triggers used by the module loader.[18:57] <WesMac> *nod* - as it is /now/, we really have no dependency management at all -- and a wide-open ecosystem (instead of a captive group of widgets or whatever)[18:57] <WesMac> BTW, feel free to jump into the newsgroup with relevant real-world experience. :)[18:57] <ls_n> For YUI, because all of the stuff we make is wrapped by our build process (just as a matter of process, not req), we can easily update the ant process to wrap module code in the diff syntax. The issues left over are the "bonus features" we've bundled in.[18:57] <ls_n> Belieeeeve me, I've wanted to be engaged in the conversation for a long time now :)[18:58] <ls_n> Stupid day job[18:58] <ls_n> I appreciate the real time conversation to help me catch up. Faster than reading 646 passionate posts :)[18:59] <WesMac> I hear you on both fronts![18:59] <WesMac> Just think, some of those email I wrote literally took hours[18:59] <ls_n> FWIW, at YUIConf, I'll be including some pimping of CommonJS and a call for engagement in my talk.[19:00] <ls_n> yeah[19:01] <WesMac> Beauty, eh? BTW, it may in the future be possible to create YUI to CommonJS shims if this next module spec is done carefully enough[19:01] <WesMac> Which would maybe be neat, I'm not sure if it makes sense to do so or not[19:01] <WesMac> But inter-system shims are a definite advantage of a wrapped format[19:02] <WesMac> .... if (!YUI) YUI = { add: module.declare }[19:02] <ls_n> It might be easier for us to just produce a parallel module suite in the other syntax, but eh.[19:03] <ls_n> The devil is in the details, there, no doubt. Since the mindset seems to be the same.[19:04] <WesMac> For sure - any work you guys do, though, let us know[19:05] <ls_n> Has the idea of "provides" been discussed? I think dojo does or did that in their code breakup/modules?[19:05] <WesMac> Yahoo has definitely contributed toward the advancement of ECMAScript programming techniques[19:05] <WesMac> ls_n: not sure - can you explain further?[19:05] <ls_n> I love the idea that multiple modules from disparate sources could provide X API that would satisfy the deps for a given module.[19:06] <WesMac> that is certainly intriguing[19:06] <WesMac> a small percentage of that problem space can be solved with computed dependencies[19:07] <ls_n> So for example, most YUI modules use some language tests currently packaged in the Lang module (part of core). So isString(x), isArray(x). But really, the modules should be decoupled from the implementation. If dojo, or some other module suite provides those APIs, that should be sufficient.[19:08] <ls_n> And yeah, to further your point about the computed deps, it would help eliminate duplicate APIs[19:08] <ls_n> because if two modules provide the same API, then one could be omitted.[19:09] <ls_n> Really, that's a whole new can of worms, though. There are a lot of directions that discussion could go.[19:09] <WesMac> absolutely. A single CommonJS module could be cleverly written with computed depends to provide APIs in terms of other libs, then used as kind of a freaky vtable[19:10] <ls_n> Or granular deps. Instead of depending on a module, specify a dep on, say, module:isArray[19:11] <ls_n> so a module that provided only isArray could be used, even if its host module didn't contain 1:1 parity with the module originally imagined.[19:11] <deanlandolt> ls_n: so you're talking about a standard for interfaces?[19:11] <ls_n> in essence[19:11] <WesMac> ls_n: that's a pretty cool idea[19:11] <deanlandolt> that is pretty neat[19:12] <deanlandolt> i've always thought things like serialization and encodings should have a common interface if possible[19:12] <deanlandolt> data access too[19:13] <ls_n> The required infrastructure to facilitate something like that would either have to be super elegant (aka small) or supplied by SS functionality, though.[19:13] <deanlandolt> i'd think SS in a build step[19:14] <deanlandolt> but more importantly i think the big win is just in documentation[19:14] <deanlandolt> a provides array of strings === 1000 words[19:14] <ls_n> Even in its current incarnation, the YUI module loader having to be downloaded to the page and process the dep tree in client space is taxing. It's pretty fast because we frontload the module dep tree, but it's still far from ideal.[19:14] <ls_n> deanlandolt: totally[19:15] <WesMac> ls_n: You want to contribute to Firefox script byte-code caching![19:15] <deanlandolt> writing to a well-defined api means you just have to document the delta between your implementation and the api...win for all[19:16] <ls_n> I think there's a lot of win that can happen by integrating a SS process, be it build step or smarter module delivery system. But I think it's also important to preserve the simpler use case, like you mentioned WesMac where the modules have to be delivered by a simple HTTP server.[19:16] <WesMac> It is /theoretically/ possible to attach compiled bytecode to the HTTP cache for scripts, then re-run the byte code when you 304[19:16] <ls_n> hehe. SCIENCE![19:16] <WesMac> This works with Firefox because the bytecode is essentially the JS AST, and it saves you from reparsing the source code (although the method JIT still has to recompile te byte code)[19:17] <WesMac> Chrome, I don't think so, I don't think they can generate anything like PIC assembly off their AST[19:17] <WesMac> great, now I have that stupid 80s pop hit stuck in my head. ;)[19:18] <ls_n> My work here is done[19:20] <ls_n> Seems to me that it would be wiser to let the browser vendors optimize patterns built out of the specified tech rather than explore things like 304 bytecode payload voodoo. There's room for innovation before crossing the border to hacking.[19:21] <ls_n> But I can be a luddite.[19:22] <ls_n> Well, I've gotta get back to prepping for YUIConf. Thanks again for the discussion. I'll try not to be a stranger :)[19:24] <WesMac> That's not hacking, at least IMHO. They already know 304, they already have stored bytecode for fastloading extensions. I personally believe that not reparsing stuff like google gears would be a measureable gain (it just wouldn't show on, you know, speed benchmarks)[19:24] <WesMac> and, yeah, stick around, nice visit![19:25] <deanlandolt> a++, would chat again!!!1[19:25] <WesMac> LOL[20:00] <WesMac> module.declare and module.provide interact in unanticipated ways[20:00] <WesMac> module.provide(["a"], hello)[20:01] <WesMac> a.js has module.declare(["b"], stuff)[20:02] <WesMac> to avoid blocking, module.declare must know to kick off hello after async-ly providing b[20:02] <deanlandolt> circlicious[20:02] <deanlandolt> maybe that's just a restriction of module.provide[20:03] <WesMac> well, yes and no[20:03] <WesMac> without module.provide, we kick off as a sync-mode environment, so we don't have to worry about the callback because it's not needed[20:03] <WesMac> this is hurting my brain[20:07] <deanlandolt> my brain is not big enough to hurt because the problem space doesn't quite fit it in :-/[20:21] <WesMac> I think the key is that the primordial module's module.declare can block[20:22] <WesMac> at least on the server[20:22] <WesMac> let's see on[20:22] <WesMac> the client, with multiple module.declares inject with multiple script tags[20:23] <WesMac> the primordial module's module.declare would have to queue up calls to require.import and inject dependencies with script tags into the core document[20:24] <WesMac> then the "real" code would still kick off based on body.onload[20:24] <WesMac> and lazy-loaded deps would use module.provide, so I think we're all good[20:25] <WesMac> module.provide could either change the character of the delivery (e.g. go full-on async with XHR)[20:25] <WesMac> or rig module.depend to inject more dependencies with script tags and kick off the callback when the script tags were done loading[20:33] <WesMac> deanlandolt: module.deps('a') vs. module.deps.a for named dependencies[20:33] <ls_n> I'm not sure if the 'provide' you're describing is seeded by our conversation, but if so, what I actually had in mind was something more like module.define(id?, [deps], factoryFn, [provides]) or module.define(id?, factoryFn, { requires: [deps], provides: [API detail] });[20:34] <WesMac> deanlandolt: the function form eliminates the need for getters or premature exporting (whihc can cause cycle wierdisms)[20:34] <WesMac> ls_n: different provide, this is more like require.async() or require.ensure() if you are familiar with that[20:34] <ls_n> k cool[20:34] <deanlandolt> hmm...i'm indifferent so if the fn form works that's great[20:34] <WesMac> ls_n: module.provide([deps], callback)[20:35] <deanlandolt> though WesMac that is a good point -- dojo's provide does something a little different so it may be an overloaded term[20:35] <WesMac> deanlandolt: *nod* - I need a word which means "make available to the environment" or something[20:35] <WesMac> I'll finish with .provide and then let the group bikeshed :)[20:35] * deanlandolt thinks[20:35] <deanlandolt> yeah, sure enough...just another note[20:35] * WesMac nods'[20:36] <WesMac> deanlandolt: anticipated used case: body.onload =function[20:36] <WesMac> deanlandolt: anticipated used case: body.onload =function() { module.provide(["a", "b"], myCallback) }[20:37] <deanlandolt> yeah, require.ensure[20:37] <WesMac> a big thing missing from the browser side is a portable way to launch a program! This fixes that while also providing explicit lazy loading of modules[20:37] <deanlandolt> just on module, because that's where it belongs...and not `ensure`, because that's confusing as sin[20:38] <WesMac> yep[20:39] <WesMac> the trick now is to make that ^^^ form work with those modules calling module.declare() which in turn may have other dependencies, and making sure we kick off myCallback when we're all done[20:40] <WesMac> I think the key is to return the dependencies from module.declare, and resolve the dependencies with module.provide so that the callback gets run at the right time[20:40] <WesMac> so this gets us syntax that works the same with either concurrency model[20:45] * WesMac looks for Gildenstern
Logs by date :