Mochabot log - CommonJS IRC channel: #commonjs on irc.freenode.net

2010-04-19:

[11:40] <kuya> ashb: hi - is db.eval supported with your mongo driver?
[11:48] * kuya rephrases
[11:49] <kuya> ashb: can you make db.eval work? :]
[12:57] <ashb> kuya: umm hmm might be possible if i've exposed run_cmd
[12:59] <ashb> seems i have.
[13:00] <kuya> oh i was grepping for eval...
[13:00] <kuya> ill look into it - thanks for the hint
[15:10] <ashb> kuya_: link issues today.
[15:10] <ashb> i think db.runCommand("$eval", "code....."); *might* do the job
[15:10] <ashb> if not, give me a failign test case >_>
[15:10] <kuya_> :)
[15:10] <kuya_> thanks ill try
[21:14] <ashb> JSON schema needs a nice clear doc. RFC is not such a doc
[21:34] <aho> ashb, http://www.json.org/ got railroad diagrams
[21:35] <ashb> aho: JSON Schema != JSON
[21:35] <aho> my bad
[21:36] <ashb> also JSON forcing quoting for simple keys was a stupid idea.
[21:36] <ashb> s/forcing/needing/
[21:37] <aho> made the spec simpler
[21:37] <ashb> its not like the rules for when quoting is needed is hard
[21:37] <ashb> if it matches /^[a-z0-9_]+$/ no quotes areneeded
[21:37] <ashb> otherwise they are
[21:38] <kriszyp> than JSON wouldn't be a subset of JS
[21:38] <ashb> thats exactly how JS object litterals behave, no?
[21:38] <ashb> js> { 1: "abc" }
[21:38] <gbot2> ashb: Error: SyntaxError: invalid label: { 1: "abc" } ...^
[21:38] <kriszyp> {if:true} would be valid JSON and not valid JS
[21:38] <aho> ashb, you cant have a number at the beginning
[21:38] <ashb> ah true
[21:38] <aho> see? complicated :>
[21:38] <ashb> js> { if: "abc" }
[21:38] <gbot2> ashb: Error: SyntaxError: missing ( before condition: { if: "abc" } ....^
[21:39] <ashb> still. it makes it a pain.
[21:39] <aho> also, yea... keywords are a no no
[21:39] <ashb> this is why i like perls => - force string on LHS
[21:39] <deanlandolt> kriszyp: isn't quoted keys optional in JSONExt?
[21:39] <kriszyp> yeah
[21:39] <deanlandolt> ashb: python's dict is like that too and sometimes it just feels clunky
[21:39] <ashb> really? why so? (i dont do python seriously)
[21:40] <ashb> only toyed with it or hacked small changes into other things
[21:40] <deanlandolt> perhaps it only feels clunky when mixed w/ javascript development (i don't remember it feeling clunky a few years ago)...but a totally subjective measure, i admit
[21:40] <ashb> kriszyp: do you know of a json schema validator that doesn't crash?
[21:41] <deanlandolt> ashb: crash? how?
[21:41] <ashb> http://james.newtonking.com/projects/json/schema.aspx
[21:41] <ashb> [NullReferenceException: Object reference not set to an instance of an object.]
[21:41] <ashb> on certain schema/input combos
[21:41] <kriszyp> http://github.com/kriszyp/commonjs-utils/blob/master/lib/json-schema.js
[21:43] <ashb> Strict warning: assignment to undeclared variable Integer at json-schema.js:21
[21:43] <ashb> btw
[21:44] <deanlandolt> ashb: he's intentionally creating a global there
[21:44] <kriszyp> hmm, yeah, I guess that shouldn't be in a commonjs module
[21:44] <deanlandolt> so that you can use Integer as your schema
[21:44] <deanlandolt> kriszyp: heh, yeah, probably not :)
[21:44] <kriszyp> or I should have be an export
[21:44] <ashb> just pointing it out
[21:44] <ashb> what you do with it is up to you :)
[21:44] <deanlandolt> exports.patchGlobalTypes
[21:44] <deanlandolt> (or something like that)
[21:46] <kriszyp> I'll just do exports.Integer = {type:"integer"}
[21:46] <ashb> what are those and the other .type used for?
[21:48] <ashb> major down side to JSON-Schema: its really verbose :(
[21:52] <jbrantly> kriszyp: to your knowledge, are there any implementations of Packages/Mappings?
[21:54] <kriszyp> I am working on one jbrantly: http://github.com/kriszyp/web-modules
[21:55] <kriszyp> ashb, the .type means that instead of writing properties: { foo: { type:"string}}, you can write properties: { foo: String } (in a javascript env)
[21:55] <kriszyp> makes it less verbose
[21:56] <ashb> good to know, but you can't have a description for the field then
[21:56] <kriszyp> yep
[21:56] <kriszyp> but if you want a description, you usually are as concerned about verbosity
[21:57] <kriszyp> aren't
[21:57] <ashb> its just a general comment of 'the schema takes up a lot of vertical space'
[21:58] <jbrantly> kriszyp: any intention of incorporating your work with web-modules with something like transporter? If so, I'd love to work with you to get that going with something like Yabble
[21:59] <kriszyp> yes, definitely
[21:59] <jbrantly> using the transporter as a proxy, for example
[21:59] <kriszyp> well, by incorporating, I don't really need to do much, web-modules exposes a getModuleSource function
[22:00] <kriszyp> so to tie the two together, you do:
[22:00] <kriszyp> require("transporter/jsgi/transporter").Transporter({
[22:00] <kriszyp> loader: require("web-modules").getModuleSource})
[22:01] <kriszyp> (assuming you have the right transporter mapped to the right package :P)
[22:02] <kriszyp> and I intend to provide a way to alternately do loader: require("web-modules").usePackage({mappings:[...]}).getModuleSource} so you could define a custom set of mappings for the browser
[22:02] <kriszyp> or something like that
[22:03] <kriszyp> need to work though how to handle overlay:{browser:{mappings:[ ... browser specific mappings ...]}}
[22:04] <jbrantly> right
[22:05] <jbrantly> is there any reason you use "foo/" instead of say "foo:" like seed.js?
[22:06] <jbrantly> seems that either would technically work, but by convention I assume you'd rather see "/". I didn't see in the spec where it was mandated.
[22:06] <kriszyp> I guess "foo:" would be fine
[22:07] <kriszyp> yeah, I just figured '/' was more appealing
[22:07] <kriszyp> but "my-package:" would be fine
[22:08] <kriszyp> that actually looks kind of nice
[22:08] <jbrantly> I like that it distinguishes between a path and a package
[22:08] <kriszyp> didn't even really occur to me that the mappings was completely compatible with seedjs's convention
[22:08] <jbrantly> indeed they are
[22:08] <kriszyp> maybe I should start using that
[22:11] <jbrantly> one last thing. I see where you have: "If a module includes a literal in a comment of the form:". There is some discussion on having a specification for module metadata, and one of my thoughts was that it could be included exactly as literals or JSON within a comment. This part of your specification may fit in nicely with the module metadata stuff.
[22:11] <ashb> i'm quite strongly against having to parse the source text of a module to get metadata
[22:11] <ashb> strikes me as a painful route to go down
[22:12] <kriszyp> ashb, you don't have to parse if you know where package.json is
[22:12] <kriszyp> (at least in the package mappings proposal)
[22:12] <jbrantly> ashb: I assume you recommend a separate file?
[22:12] <kriszyp> it is only if you request a module by url, and the loader needs to figure out what package that module is a part of
[22:12] <ashb> kriszyp: the meta proposal jbrantly refers to talks about looking for META({...}) in a file
[22:12] <ashb> which is just plain insanity
[22:12] <jbrantly> lol
[22:13] <kriszyp> ok, just defending the pacakge mappings definition :)
[22:13] <ashb> i haven't really thought/looked aover the mappings stuff
[22:13] <jbrantly> kriszyp: no, completely different
[22:13] <ashb> saw it come on the ML but thats about it
[22:13] <kriszyp> its not a very long proposal
[22:14] <jbrantly> ashb: I agree that the proposal as-is is insanity. But what do you think about the idea, in general, of metadata for a module (as opposed to a package), and would you rather see that in a separate .json file like package.json?
[22:15] <ashb> i dont see what metadata you want about a single module
[22:15] <ashb> that isn't covered by a package
[22:15] <ashb> & # sleep
[22:16] <jbrantly> ashb: the biggest thing to me, personally, would be shallow dependencies. I'm not sure if there are other advantages. Thats one of the things I've asked for in the thread; what else would/could be useful.
[22:16] <kriszyp> jbrantly: is the issue that parsing for require(moduleId) is too difficult?
[22:16] <jbrantly> kriszyp: basically, yes.
[22:17] <kriszyp> web-modules does that parsing, and I haven't found that to be difficult. It has been clearly stated in the ML (and maybe in spec) that require *must* be given a static string as its argument (if you want your module to be compatible)
[22:18] <jbrantly> kriszyp: err. I somehow missed that static string part. *goes to look it up*
[22:18] <kriszyp> yeah, I don't know if it made into any spec
[22:18] <kriszyp> can't remember, but I know there have been discussions about that
[22:18] <jbrantly> kriszyp: its not so much "difficult". I've got it in Yabble as well. But there are edge cases where it fails.
[22:19] <kriszyp> does it fail in the form of require("module-id")?
[22:20] <jbrantly> kriszyp: the other area though is require.ensure. Consider a module: require.ensure(['moduleA'], function(require) { require('moduleA'); });
[22:20] <jbrantly> moduleA is not a shallow dependency, really. But it will get flagged as one.
[22:20] <kriszyp> yeah, I have thought about that
[22:20] <jbrantly> right now I do a thing where I pull out all requires, and then all require.ensures(). Then I remove any in the second set from the first.
[22:21] <kriszyp> of course being over-zealous in grabbing too many deps never breaks anything
[22:21] <jbrantly> the reasoning being that if you do a require.ensure for a particular module, you probably didn't also do a normal require() for it
[22:21] <kriszyp> but obviously you want to defer in that case
[22:21] <kriszyp> right, I understand
[22:22] <jbrantly> so to answer your question, yes, it can fail in the form of require("module-id") when require.ensure gets involved. Otherwise its fine.
[22:23] <jbrantly> I wouldnt mind seeing the literal string part in the spec though. One less thing to worry about :)
[22:27] <jbrantly> anyways.... one of my original thoughts for fixing the require.ensure thing was some kind of proprietary way to specify shallow dependencies for a module. Similar to the stuff here: http://qooxdoo.org/documentation/1.0/code_structure
[22:28] <jbrantly> just in case you needed to fine-tune something, you could
[22:29] <jbrantly> jhuni started this Modules/Metadata proposal. He's taking it a bit further than it needs to go, I think, but I'm seeing if I can't get some kind of simple module metadata spec out of it.
[22:30] <kriszyp> yeah, I think that makes sense, as optional metadata that can be used for perf/module loading optimization
[22:30] <jbrantly> yes, one keyword there is definitely *optional*
[22:30] <kriszyp> where it doesn't normatively affect behavior, just provides information about how module load can be deferred
[22:30] <kriszyp> that seems reasonable
[22:30] <jhuni> jbrantly: in what way do I take it too far?
[22:31] <jbrantly> jhuni: I've explained that on the mailing list, I think
[22:36] <jhuni> jbrantly: So you must use require with string literals and it doesn't work after the event loop?
[22:38] <jhuni> "require() should be disallowed after the event loop starts"
[22:39] <jhuni> jbrantly: also this is what I described for using require/exports http://wiki.commonjs.org/wiki/Modules/Metadata#Backwards_Compatibility
[22:41] <jbrantly> jhuni: so you're making a CommonJS specification that can optionally use the CommonJS Modules specification for backwards compatibility? That doesn't make much sense to me.
[22:41] <jhuni> jhuni: ignore the term backwards compability
[22:42] <jhuni> jbrantly: ignore the term backwards compability
[22:42] <jbrantly> jhuni: in short, what I think you're trying to do is re-invent CommonJS. What I'm trying to do is have module metadata.
[22:42] <jhuni> if you want to you can still use require, it is just preferable not to use it for client-side applications
[22:43] <jbrantly> jhuni: sure. Thats still possible with everything as it is now using Async/A aka require.ensure
[22:44] <jbrantly> jhuni: I guess I just don't see a need to reinvent things here. The things that you're saying sound good, but they're all possible right now.
[22:44] <jhuni> jbrantly: if I come across a module how to do I tell if it is a commonjs module or just a normal javascript module?
[22:44] <jbrantly> jhuni: define "normal javascript module"
[22:45] <jhuni> jbrantly: one that doesn't have require, exports, etc
[22:45] <jhuni> jbrantly: like jQuery
[22:45] <jbrantly> then there you go, I guess. If require and exports are free variables, then there is a good chance its a CommonJS module.
[22:46] <jhuni> jbrantly: k, but it would be helpful to put a header like that one so that it is clear.
[22:46] <jbrantly> why?
[22:46] <jhuni> jbrantly: compatibility with non-commonjs modules
[22:46] <jbrantly> I'm not trying to load jquery modules. I'm trying to load commonjs modules.
[22:46] <jbrantly> jhuni: I have no interest in compatibility with non-commonjs modules
[22:47] <jbrantly> half the point is to try to get modules to be "common" :)
[22:48] <jbrantly> btw, many libs are not compatible with CommonJS. For instance, qunit does a check to see if its running in a CommonJS environment
[22:48] <jbrantly> *not = now
[22:49] <jhuni> jbrantly: but also the ability to know if a module is commonjs that way you can go through all require statements with that regex
[22:49] <jbrantly> jhuni: but I don't need to know if a module is CommonJS. I assume that it is.
[22:50] <jhuni> jbrantly: the metadata could be useful to some people who are using your code
[22:50] <jhuni> jbrantly: but also to some servers that are trying to compress it
[22:52] <jbrantly> jhuni: don't get me wrong. I'm all for metadata. That metadata may even specify what version of CommonJS Modules that module is compatible with, for example (thus solving your "is it a commonjs module"). But what you're proposing is not module metadata. It's module metadata plus a new loader plus a use server, etc, etc.
[22:53] <jhuni> jbrantly: It allows for compability though, you can just execute the scripts as is, and if you want place metadata to tell people what version of commonjs module system you are using
[22:53] <jbrantly> jhuni: thats what I meant by the goals need to be clearly specified and the specification scoped accordingly. The goal is to specify module metadata. The specification as it is talks about some kind of new loader mechanism, doing away with require(), introducing a new API that is similar to require.ensure, etc.
[22:54] <jbrantly> jhuni: what the specification *should* be is a format for describing module metadata
[22:54] <jhuni> jbrantly: I would like to redesign the wiki page when my implementation has matured further, when I fix circular dependencies and what not
[22:55] <jhuni> jbrantly: I personally do not intend to use require because I don't think it is good for client-side performance
[22:56] <jbrantly> jhuni: then use require.ensure, not some new made-up thing like META.getJS
[22:58] <jhuni> if I want to take a commonjs script and turn it into something that is client-side friendly is there any function like META.getJS that does that?
[22:58] <jbrantly> require.ensure?
[22:58] <jhuni> require.ensure does combo handling?
[22:59] <jbrantly> you can do require.ensure(['moduleA', 'moduleB'], function () {})
[22:59] <jbrantly> how those modules are actually loaded depends on the loader
[22:59] <jhuni> Yeah, but it doesn't take a commonjs script, get its dependencies, and include them in the file
[23:00] <jbrantly> jhuni: neither does your script. That's something for the server-side. We were just talking about kriszyp's transporter which does just that.
[23:00] <jhuni> META.getJS turns a script into something client-side friendly
[23:00] <jbrantly> uhm
[23:01] <jhuni> if a file has a bunch of "use" statements in its metadata, META.getJS returns the JS of that file with all of its things included
[23:01] <jhuni> then the loader can eval that file
[23:01] <jbrantly> Yabble does something similar
[23:01] <jhuni> Is there a Yabble.getJS function?
[23:02] <jbrantly> if a module has a bunch of require()s, those dependencies are loaded first, and then the original module is run once the dependencies have been resolved
[23:02] <jhuni> does it make it client-side friendly though?
[23:02] <jhuni> by making it into one big file?
[23:03] <jbrantly> how is that "client-side" friendly?
[23:03] <jbrantly> you're doing it all on the client
[23:03] <jhuni> client-side friendly because when everything is in one file then it takes only a single http request to get it
[23:04] <jbrantly> in your script right now, it does multiple requests
[23:04] <jbrantly> you can't turn it into a big file on the client
[23:04] <jbrantly> that doesn't make sense
[23:05] <jhuni> jbrantly: it uses the same function on the client and the server
[23:05] <jbrantly> now we're getting somewhere
[23:06] <jhuni> jbrantly: The point is, if I have program.js like in Modules/1.1.1 and it requires increments.js which requires math.js, is there a function out there which will
[23:06] <jhuni> take all those files and combine them?
[23:07] <jbrantly> there are things available to do that. See http://github.com/kriszyp/transporter for example. Yabble also ships with yabbler.js which will eventually do that, but as a build-step.
[23:07] <jbrantly> so yes, you can do that
[23:07] <jbrantly> there are tools to do that
[23:07] <jhuni> jbrantly: I am wondering if there is such a function
[23:08] <jhuni> jbrantly: a specific function like META.getJS
[23:08] <jbrantly> if you're asking whether or not my loader can be run server-side and do script-serving, then no. But it doesn't need to.
[23:09] <jhuni> jbrantly: I would just like some sort of playground where I can see how commonjs modules are optimized, for example if a require statement is in an eval block?
[23:09] <jhuni> jbrantly: if there is a require statement in an eval block you can't get its metadata so you must do a synchronous request?
[23:11] <jbrantly> you could see my unit tests, I suppose
[23:12] <jhuni> jbrantly: also if I have require(isTrue ? "module1" : "module2") doesn't that mean I got a sync request?
[23:13] <jhuni> jbrantly: which would be a case where that script would not be something in which I want to use in the client-side
[23:13] <jbrantly> jhuni: I was just talking to kriszyp and apparently string literals are required in require() statements, but its not well documented, or something. But I agree, that is a potential area where its difficult to determine shallow dependencies through static analysis and where metadata would be helpful. But in my world, what would happen in that situation is that both module1 and module2 would be dependencies, and they would both be loaded p
[23:15] <jbrantly> keep in mind, I said metadata would be helpful, not that require() needs to go away or that a new API needs to be introduced
[23:15] <jhuni> jbrantly: okay, but it is something I wouldn't use for myself
[23:17] <jhuni> jbrantly: so if you wanted to make this system more commonjs friendly you would probably want to list all the cases require can be used in an array?
[23:17] <jhuni> jbrantly: perhaps a "load": [] option?
[23:17] <jbrantly> jhuni: right. Modules that should be loaded *before* this module is executed
[23:18] <jhuni> jbrantly: okay, I suppose that works, but the require statement is still way too confusing, you can't use string literals with it and is it really getting a module? what is it doing?
[23:19] <jhuni> jbrantly: anyways you have probably heard all my reasons why require is bad already =/
[23:19] <jbrantly> sorry for the confusion. You *have* to use string literals with it
[23:19] <jhuni> jbrantly: but also the issue of is it really able to get a module for you from external locations?
[23:20] <jhuni> jbrantly: perhaps it is better named cache[] to indicate that it is getting a module that is already installed =/
[23:20] <jbrantly> in Yabble, I do something kinda like what Ryan was saying for Node.js. Basically, I load the entire application (all of the dependencies which are synchronously require()d) before executing the first one. Or something similar to that.
[23:20] <jhuni> jbrantly: anyways enough require bashing, if a bad thing has to exist I think I can live with it
[23:20] <jhuni> lol
[23:21] <jhuni> jbrantly: so always use require with string literals, and don't use it with eval
[23:21] <jhuni> jbrantly: there are probably other cases that you shouldn't use it as well
[23:22] <jhuni> jbrantly: perhaps you shouldn' t use it at all!
[23:22] <jbrantly> jhuni: eval stuff might still get picked up, although that might actually be an error
[23:23] <jbrantly> jhuni: actually, I think that for most webapps its good to use it
[23:23] <jhuni> jbrantly: you can actually pick up metadata in the META() function by using javascripts builtin interpreter
[23:23] <jhuni> jbrantly: don't even need a string processor
[23:24] <jbrantly> jhuni: because in most webapps, everything is "compiled" into one file in the end anyways. In other words, the whole app is loaded anyways. You only want to use require.ensure when you want to do lazy-loading of modules
[23:26] <jhuni> jbrantly: you can get the META data by executing the script using JavaScripts builtin functionality and then saving the metadata somewhere, like META.lastMetadata = arguments[0]
[23:27] <jbrantly> jhuni: the problem with that is that you would need to execute the module before reading the metadata. I need the other way around
[23:27] <jhuni> jbrantly: also if the issue is getting the Metadata loader/dependencies system to work on some implementations, all it needs is asynchronous slurping
[23:29] <jhuni> jbrantly: you can get the metadata and just not use the scripts return value
[23:29] <jhuni> jbrantly: but yeah, I think the current option works fine
[23:31] <jhuni> jbrantly: I will try to allow compability with the current codebase as much as possible, however, if the current loader system is flawed maintaining as some sort of "standard" is not a good idea
[23:37] <jbrantly> jhuni: sorry for the delay. I'm all for trying to fix a flawed standard. The thing though is that its really not too flawed. And small, incremental changes seem to be the way to go around here.
[23:38] <jhuni> jbrantly: sorry I realize that that may have been a bit insensitive, one of the reasons for using metadata as a loader is the metadata system is built with high-performance client-side websites in mind
[23:39] <jhuni> jbrantly: where as I feel the require statement has server-side users in mind, and the whole client-side thing is an afterthought
[23:40] <jhuni> jbrantly: it was built to make javascript act like perl or python
[23:41] <jbrantly> jhuni: yes, but we must play nicely with our server-side friends :) They did rename from ServerJS to CommonJS just for us, you know
[23:43] <jhuni> jbrantly: Do you have any hints on how the specification should be rebuilt?
[23:43] <jhuni> jbrantly: I am thinking of getting rid of the Use Server part or move its content to other areas
[23:49] <jbrantly> jhuni: frankly, I think there needs to be more discussion on it first. Perhaps the wiki can be updated to have a couple different options and explain some things like the goal, the problem(s) being solved, and potential solutions.
[23:50] <jbrantly> jhuni: but the *biggest* thing is that it needs to be scaled waaaay back. To me, it just needs to be a way to specify metadata.
[23:50] <jhuni> jbrantly: the accesibility and synchronicity things are a bit of a description of some of the goals
[23:52] <jhuni> jbrantly: my goal is relatively simple though, further the use of metadata to describe JavaScript modules, and ideally list your dependencies data as metadata as well
[23:59] <jbrantly> jhuni: so a couple of key questions are: What data exactly is available in the metadata. We've talked about dependencies. What else? Each item really needs to be useful and well thought out.

 

 

Logs by date :