Mochabot log - CommonJS IRC channel: #commonjs on irc.freenode.net

2010-07-07:

[15:21] <Wes--> Dantman: ping
[16:40] <Dantman> Wes--, if it's about the website... I haven't been able to figure out the issue... It's not apache (kommonwealth still works), it's not memcached (disabling it doesn't fix anything), and I can still connect fine to mysql... Looks like something is broken with all php on that server...
[16:56] <Dantman> This'll all be fixed if I can figure out how to keep my personal servers physically automatically backed up without using any of redwerks' machines
[17:46] <Wes--> dantman: want a hand troubleshooting?
[17:46] <Dantman> I've got work to do right now, I'll have to do that later...
[17:47] <Wes--> Dantman: 'k - ping me whenever (or call my voicemail, it tracks me down) -- I have time+experience to throw at this to try and help out
[17:48] <Dantman> I still eventually want to move commonjs to my own servers once I figure out how to back them up...
[17:48] <Dantman> IMHO, aws is shit...
[18:02] <WesMac> Dantman: how about you keep a private machine here? I could give you, say, a linux server at the end of a T1. You run it, put a CommonJS mirror on it, I'll have my ops guys monitor it.
[18:03] <Dantman> I'm trying to keep more than just commonjs backed up...
[18:05] <Wes--> dantman: as long as you don't saturate my T1 in the day time, I don't care. :) (yes, I know a T1 is not much bandwidth, but it's what I have in my office -- latency for ssh is really good)
[18:11] <WesMac> Dantman: as a trouble shooting step - your server's problem is happening between the HTTP request and the returned header. I get ~45s requests, but over 300 KB/s when it's actually transferring data
[18:12] <Dantman> It's somewhere in php as far as I can tell
[18:14] <Dantman> Bleh... stupid Zip STORED method... I'm just going to compress everything I export.
[18:15] <WesMac> gzip!
[18:23] <WesMac> Dantman: that's not a bad conclusion, I wonder why PHP is so angry? -- bad DNS lookup for logging? full session directory?
[18:25] <Dantman> Ugh... why is phpMyAdmin working now? It was broken last time I checked...
[18:25] <Dantman> Now it sounds like there IS something causing a MediaWiki specific issue
[18:26] <Dantman> Ok, it looks like the wiki IS also /working/ but in a troublesome slow way.
[18:26] <Dantman> Hmm, phpMyAdmin is slow too
[18:26] <Dantman> But not as slow
[18:36] * Dantman wonders if he should setup a small form low energy box
[18:43] <Dantman> If it had a SSD I'd consider setting up something like http://ncix.com/products/index.php?sku=48946&vpn=MAGHD-NS01-U&manufacture=Zotac
[18:56] <zumbrunn> Dantman: how about a rackspace cloudserver instance dedicated for commonjs.org
[18:57] <zumbrunn> where anybody that feels to help running it could have root access
[18:57] <zumbrunn> we could provide that
[18:58] <zumbrunn> and it comes with daily and weekly backups
[18:58] <Dantman> Ugh... an entire server for one wiki feels like a waste when there are other wiki to be run that can be on the same server.
[18:58] <WesMac> it's only a waste if it's a physical server
[18:58] <Dantman> waste = money waste
[18:58] <WesMac> are those really only $11/month?
[18:58] <zumbrunn> and several people of us can pull backups via rsync as we feel it's needed
[18:59] <Dantman> More like $14/month, smallest size
[18:59] <WesMac> dantman: I've burned about $1200 of billable time working on a CommonJS.org mirror so that I can look up specs locally
[18:59] <WesMac> money isn't the issue by any stretch
[19:00] <WesMac> $14/month, PageMail can pick that up if we can pay by the year
[19:00] <Dantman> monthly
[19:04] <WesMac> monthly payments *suck*
[19:05] <WesMac> Of course, the real expense for this type of stuff the guy that will watch it and do the grunt work
[19:05] <WesMac> I didn't volunteer precisely because I haven't got the bandwidth to be that guy
[19:13] <zumbrunn> well, just let me know... if you like, I can provide you with an instance, with billing running over the JS-server-side group (a non-profit organized as a swiss verein)
[19:14] * Dantman already has rackspacecloud
[19:16] <mikeal> we shouldn't have to pay
[19:16] <mikeal> anywhere we want to host will most likely offer free hosting
[19:16] <mikeal> we just need to find somewhere and have someone be in charge of it
[19:16] <mikeal> i'm getting questions about the traffic load on commonjs.org
[19:16] <mikeal> what kind of traffic do we have typically?
[19:19] <Dantman> I never got to setting up stats on voltaire
[19:19] <funkatron> badonk
[19:22] <funkatron> ping mikeal
[19:23] <mikeal> hiya
[19:23] <mikeal> we don't have stats on traffic :)
[19:23] <Dantman> *sigh* I wonder if I should pull 300 out of my savings and get one of these to keep stuff backed up http://ncix.com/products/index.php?sku=48946&vpn=MAGHD-NS01-U&manufacture=Zotac
[19:24] <mikeal> why don't we just move this all to the IETF :)
[19:24] <funkatron> mikeal: gotcha
[19:24] <mikeal> start a working group
[19:24] <mikeal> their tools seem to stay up
[19:25] <Dantman> backups <=> online
[19:26] <funkatron> mikeal: I dunno that I'd be the best option -- I'd think some other folks could hook you up with better stuff -- but I would be willing to provide hosting on one of my Linode VPSes
[19:26] <mikeal> i mean, if that's what we were going to do I would just want Linode to sponsor a VPS
[19:26] <mikeal> and put someone in charge of it
[19:26] <funkatron> right, and I bet they might be willing to do that
[19:26] <funkatron> then you could set it up however you want/need
[19:27] <funkatron> or you could find someone who *would* be willing to (maybe Site5 or Rackspace)
[19:28] <funkatron> I could prob poke around if you don't have contacts at these places
[19:31] <mikeal> that would be cool
[19:34] <funkatron> lemme shake a couple trees
[19:36] <funkatron> do you have a feel for the kind of box you'd need? would, say, a 512MB VPS work for you?
[19:37] <mikeal> so
[19:37] <mikeal> I really don't want to start any kind of discussion about change tools or wikis
[19:37] <mikeal> cause that will just expode
[19:38] <funkatron> mmmhmmm....
[19:38] <mikeal> so, whatever the current data and tech is I just want to hsot that
[19:38] <funkatron> I don't know what the current data/tech was hosted on
[19:39] <Dantman> an aws server from work...
[19:39] <Dantman> The one we actually keep physical backups of
[19:40] <funkatron> well, I'd imagine you could run a wikimedia installation in a 512MB VPS Lamp stack setup unless it was getting hit hard
[19:41] <Dantman> s/wikimedia/mediawiki/
[19:42] <WesMac> Dantman: something new just happen over there? I can't even pull PNGs from you now
[19:42] <mikeal> anyone know if it's MySQL or Postgres?
[19:43] <Dantman> mysql
[19:43] <funkatron> I keep saying that, sorry
[19:43] <funkatron> derp
[19:43] <Dantman> WesMac, apache still running, dumps still accesible for me
[19:44] <WesMac> Dantman: Hm, wierd. How about http://commonjs.org/images/1/19/Header_wrapper_bg.png ?
[19:45] <Dantman> hmm
[19:46] <Dantman> Dumps, but not images? O_o
[19:47] <WesMac> images were loading okay eariler, which is even wierder
[19:47] <funkatron> hey guys, preinheimer_ is a friend of mine, longtime FOSS community guy, and he runs a system called WonderProxy. He's interested in sponsoring hosting
[19:47] * preinheimer_ waves
[19:47] <funkatron> so he's here now, and you should talk to him about your needs and what you need to get up and running again
[19:48] <preinheimer_> Should I be speaking to everyone, or some subset of the group that's handling your site?
[19:48] <WesMac> preinheimer_: Dantman has the keys
[19:49] <preinheimer_> Do you know much about what you need? I'd suggested a linode 512
[19:49] <Dantman> Never needed anything special for a MediaWiki install besides a minimal vps/cloudserver
[19:50] <preinheimer_> I'm hoping for something nice like "Hosting graciously provided by Wonderproxy" on your site.
[19:50] <preinheimer_> Cool, it seems like that should be more than enough then.
[19:50] <preinheimer_> (btw http://wonderproxy.com I'm not evil or anything, also: not porn)
[19:50] <WesMac> preinheimer_: I suspect, FWIW, that the largest missing resource here is "time to fix/setup software". Unfortunately -- because time is universally the hardest resource to come by
[19:51] <Dantman> WesMac, oh, anything commonjs.org goes through php
[19:51] <Dantman> http://wiki.commonjs.org/images/1/19/Header_wrapper_bg.png
[19:51] <preinheimer_> Presuming you're okay with debian, I'm happy to get php, apache, mysql set up for you
[19:51] <WesMac> dantman: ah, argh
[19:52] <WesMac> dantman: are those image dirs the same on both vhosts?
[19:52] <preinheimer_> I'll even run through a vanilla mediawiki install
[19:52] <Dantman> commonjs.org is a php script that fetches stuff from the wiki... can't remember how I did static images
[19:54] <Dantman> WesMac, looks to me that I wrote it so commonjs.org points directly to the files on wiki.commonjs.org, it doesn't do any finicky things doing images on commonjs.org
[19:55] <preinheimer_> I need to go home (I'm around the corner in a coffee shop) bbia 10
[19:55] <WesMac> dantman: yeah, that seems to have helped
[19:55] <WesMac> preinheimer_: ttys :)
[19:55] <Dantman> preinheimer_, you keep physical backups of the server?
[19:58] <mikeal> there is all kinds of magic in the wiki
[19:58] <mikeal> it took me like an hour to figure out where things were the first time i did edits
[20:01] <Dantman> server with backups, or wiki online?
[20:02] <Dantman> preinheimer_, you keep physical backups of the server?
[20:02] <preinheimer_> personally
[20:02] <preinheimer_> ?
[20:02] <preinheimer_> I back up most of my servers to a second drive in another server
[20:03] <funkatron> doing a setup with rsnapshot isn't too hard and does a pretty solid job, even from a machine at home
[20:03] <funkatron> that's how I do my backups, but I have a redundant one with linode as well
[20:03] <preinheimer_> I think our billing records are dup'd to my sys-admins home machine
[20:04] <Dantman> The wiki is currently on the server that gets backed up to a machine running 24/7 in the office.
[20:04] <Dantman> I haven't found a good method of backing up my personal servers yet
[20:04] <Dantman> I don't have any machines in my house that I run 24/7
[20:05] <Dantman> And I don't want to run my loud desktop 24/7 either
[20:05] <Dantman> Where the external drive (where I backup my laptop) adds to the sound
[20:07] <preinheimer_> Having multiple personal servers helps the backup situation. Everything is on at least two drives for me right now.
[20:07] <preinheimer_> there is a SPF in the machine with the dual drive, but other than that I think I'm pretty good
[20:07] <Dantman> I don't like to rely on remote machines for backups
[20:07] <funkatron> hrm
[20:07] <funkatron> if you're not willing to run a local machine 24/7, what will you rely on then? 8)
[20:08] <preinheimer_> fast recovery. Unless your home pipe is faster than my co-lo'd net connection
[20:08] <Dantman> When it comes to backups, I value reliability of the data staying there, over connection speed...
[20:09] <funkatron> in that case, it sounds like you should sort out your own backup solution you can host locally
[20:09] <Dantman> funkatron, The issue is not wanting to run no machine 24/7, it's having nothing I want to run 24/7... I don't have any nice quiet low-form eco pcs
[20:10] <funkatron> but, I think preinheimer_ is offering hosting for you guys, if you need it
[20:30] <Wes--> Dantman: Hopefully the irony of this situation is not lost on you -- since you have no backup solution, we have no spec wiki at all
[20:31] <ashb> Dantman: the best backup solution is one that works.
[20:32] <Dantman> The one that works needs a machine I don't want to buy till my next paycheck
[20:32] <ashb> then it doesn't work.
[20:32] <Wes--> Dantman: You don't need to fly this task solo. This is a community.
[20:32] <ashb> it also isn't a backup unless you've actually restored data from it. its just hope in that case
[20:34] <Dantman> *sigh* What did I put on tama again?
[20:35] <Dantman> hmmm... perhaps I'll tama=dns, dnsui, svn, viewvc; kama=mediawiki hosting dev; and setup a newer server for wiki
[20:37] <Wes--> Okay, anybody else needing specs, you can get a copy here: http://www.page.ca/~wes/cjs/wiki.commonjs.org/
[20:37] <Wes--> Let me know if you find any critical oversights
[20:43] <Wes--> heh, nothing like a bunch of people hitting your site to show you 404s for stuff you missed :D
[20:44] <Dantman> ;) That's why I intent to give hosted Kommonwealth sites the ability to statistically log 404s and present a list to the person making the site.
[20:45] <Dantman> @_@ Ah right... firewall setup again...
[20:54] <Wes--> can somebody tell me how to tweet to commonjs?
[20:57] <Wes--> or does twitter not work like that?
[20:57] * Wes-- is clueless when it comes to technology aimed at 14-year-old girls :p
[21:04] <Dantman> Someone controls the user account @commonjs... don't know who
[21:04] <Wes--> Oh, so I can't send a message to everybody who is interested in commonjs?
[21:06] <Dantman> Not using the @commonjs user account
[21:06] <Dantman> .....I suppose there's the #commonjs hashtag... I never really use those
[21:07] <Dantman> O_O nano...... has colouring?
[21:14] <mikeal> dammit
[21:14] <mikeal> i actually need to make edits to the Packages spec before i can put it up for a vote
[21:16] <Dantman> The wiki isn't exactly down... it's just responding very badly actually
[21:19] <mikeal> i didn't their could be a less reliable wiki than the Apache CouchDB wiki, but i have been proven wrong :)
[21:19] <mikeal> actually, this is still less annoying
[21:19] <mikeal> because the couchdb wiki just gives you 500s half the time you try to edit pages and then you can't recover your edits :(
[21:19] <Dantman> Blame old aws servers setup by other people which have overstayed their lifespan
[21:20] <mikeal> i don't want to blame anyone, i just want to find a sustainable solution
[21:21] <Dantman> Temporarily, how about grabbing the page out of the pages.xml dump and saving it to http://nadir-point.com/wiki/Packages temporarily.... though I can't guarantee how long that wiki will be reliable for
[21:21] <Dantman> That wiki is running on my ancient linode vps
[21:23] <mikeal> if ryan wasn't on vacation i could see if he could get us some free space on joyent
[21:25] <Dantman> blame my tendency to hold of on mentioning something till its already finished for not mentioning this yet... but I'm setting up another server for wiki, even though it won't have a backup setup yet...
[21:29] * Dantman should setup puppet on the backup machine when he sets on up...
[22:46] <Dantman> It's probably not going to matter since it is almost inaccessible, but I'm going to make the wiki read-only
[22:53] <mikeal> who do we know at media temple?
[23:13] <JohnnyL> what's the time like looking at a js file aware system?
[23:13] <JohnnyL> time line
[23:29] <JohnnyL> YOU ARE ALL FIRED!

 

 

Logs by date :