Tuesday, November 02, 2010
It's also just over 5 years since I started out working on my own, and a year since I started working again with a couple of old friends. One of those (Reema) has just found herself a real job so she'll be leaving us and I wish her well. Starting new things is usually harder and slower than I imagine, which is good, since I'd probably not bother otherwise.
At the machine level: we've just brought online a new server and moved all the services left on my oldest server in preparation to shut it down. Shutting down old servers is rarely worth it from a time/finances point of view, but it seemed like the right thing to do in this case, in spite of the backlog in client work I've now accumulated.
And of course, in the global scheme of things, death has the excellent function of making space for new things, and that's very true in this case. My new server is using pressflow and varnish to be super scalable for a couple of new big sites, one of which is already humming along on it (http://socialinnovation.ca). Rob Ellis has also installed apache solr on it, and it's a huge relief to be able to use it on my larger sites: compared to stock Drupal search, it gives better results, it's faster, and the site database backups are about half the size (having the search database being backed up always seemed ridiculous anyway).
Monday, October 25, 2010
Then Kahlid asked me to come do a real presentation at their Waterloo group. That presentation is now up here:
After my presentation, Khalid said he was hoping for more details about the Features module, so when I was thinking about what I could offer to Toronto Drupal Camp, I decided on a straightforward intro/presentation. That went surprisingly well (considering that I don't usually do "straightforward" very well), and now I've ended up feeling like a Features ambassador, evidenced by the following diplomatic photo:
Thanks to Chris Luckhardt
Short answer: I use Features on all my new projects of any size bigger than tiny and encourage other developers to also. It's not (yet?) the holy grail, but it's definitely an essential tool for managing the ongoing development of any site. Want to know more? Check the links off the drupal camp session description
Monday, August 30, 2010
I'm currently on hold trying to unravel their latest, truly, incredibly, bad business practice.
It started earlier this year, when I tried to combine my bills that had been up until then separated. I didn't care too much except that I would get these irritating advertisements for cheap internet and when I phoned them up, they'd tell me I wasn't eligible since I was already a customer.
After combining my bills (that was the only way they could give me a cheaper service), it got worse and the last straw was when they charged me $.20/minute for a call to Spain. Apparently not covered with my normal long-distance plan to which I'd enrolled, they decided they could just charge any old extorsionist rate... But even aside from that, they had made several other errors, and after sorting them all out, I was transferred to collections who told me I had to pay up right away or get disconnected.
Anyway, I go tired of it all, so I switched both services to TekSavvy (who use Bell's infrastructure) and have been very happy since.
Unfortunately, even after paying off the existing bill (unhappily), the story wasn't over. A month after quitting, I got a bill saying they owed me some money. Then I got another one saying we were even. They another month later I got one saying I owned them $30. I ignored it like a bad dream, assuming I would get another random bill the next month.
Unfortunately, if you ignore your bill at Bell, it goes to collections. That means I started getting calls to pay my bill. When I explained the situation, they told me they couldn't help me, I had to call back and ask for billing. When I did that, I followed the various prompts and ended up again at collections. The friendly guy there explained that in order to get to billing, I had to ignore all the voice prompts and just keep pressing 0 (presumably special treatment because I had cancelled my service?). I explained that this was all a little too much like Kafka and gave up in rage for the day.
Since then I've had a number of further calls from collections, all to the same effect - they can only ask me for money and can't even tell me why I'm being billed.
Finally last week I got a collections notice and today I decided I'd better just deal with it.
So today I phone up and didn't follow the voice prompts. I did tell them it was for internet, since that's what the collections person I talked to had told me. I talked to one nice person for half an hour before she told me she couldn't do anything, so she put me through to Lavinia, who is a supervisor there. After explaining all over again, she decided that the charge was actually for the phone, not for the internet, and therefore couldn't help me either, and I'm now waiting on hold for a phone service billing supervisor.
Ah, now Lisa's got the ball. She's just taken another 15 minutes of time to get up to speed and has put me on hold again. Don't you hate when they put you on hold and blast music at you? This one's a quiet hold, but the first one I talked to today apologized because she couldn't mute the music when she put me on hold, and it was twice as loud as her voice. Poor Lisa's going to go through all the bills again and try and figure out if that $30 is valid (from her accounting perspective). I can only write my blog in the interim as a kind of therapy so I don't ruin my day completely.
A few minutes later ... a final update. Lisa is back on the line and tells me the charge was for an early termination fee and that she would waive it. Now I appreciate that, but that's a nice fiction between the two of us. The truth is that this story should never have happened, but unless someone at Bell is paying attention, it'll all happen over again, again again.
Thanks Lisa and Lavinia!
Sunday, August 08, 2010
So, big thanks to the Surmans who shared their cottage rental up north of Parry Sound with us and threw in the free photos as a bonus (left, and lots more on Flickr). And also the St. Louis Andersons and Chantal and Carter, who also came with their delightful children. Special mention goes to Estora for coming blueberry picking with me and the dogs before anyone else was up, and to Kelly for making the yummiest blueberry and peach pie that ever was.
And, having said that, as I go through my inbox and listen to the radio to reacquaint myself with the human part of this country that I live in, I also want to give thanks to a couple of people who I think are responsible in a kind of fundamental way for these gifts that we all enjoy:
I just heard his newish show "the bottom line", and his interview with a BP oil executive on the radio this morning has me completely impressed with his combination of scientific understanding, political savvy, and above all his courage.
Also still going strong in her 90s now, she talks about the erosion of democracy on The Current. If you prefer print, try this article by Lawrence Scanlan. I generally scoff at such stuff as partisan fear-mongering, but I think it's time I woke up out of my comfortable indifference. I'm still impressed with Stephen Harper's political skills, but no longer apathetic about what he's been up to with his minority. There's going to be an election sometime in the next couple of years, and the law of averages suggests sooner rather than later, and I think it might be time to call a spade a spade and see clearly just what kind of a leader we've got. It's not about your choices anymore, it's about whether you want a choice. We're not shopping for a suit, we're being asked if we think democracy matters between elections.
Stephen: the answer is yes.
Saturday, July 24, 2010
1. Toronto Workforce Innovation Group
This was a project of Blackfly, the collaborative whole-site web development shop I've been working on since last fall. Most of the work of this site was done by my colleague Reema. It's a great showcase of how you can assemble some pretty standard parts, and with a minimum of coding, create a great looking, functional site that still reflects the complexity and specificity of an organization. She made good use of a theme from TopNotchThemes, and the interesting relationships between the "Challenges" section and the "Solutions" section was a nice text-book use of views/cck.
2. Rosenberg Foundation for Children
This is a small non-profit based in Western Massachusetts, started by the younger son of the Rosenbergs who were executed in the US in the 50s for treason. They're celebrating their 20th anniversary this year of supporting children of activists. I don't usually work with organizations in the US, just because of the complications of distance and sometimes culture, but I really liked the mission of this organization, and my contact there is super nice, and they came looking for me (because of my CiviCRM expertise), so I didn't want to say no. They had been suffering from the inadequate service of a previous provider, so it was a kind of a rescue project - extracting them from the previous server and all it's out-of-date modules and badly implemented features. They're now happily settled into a local (for them) service provider called Florence IT, who I've been pleased to get to know and work with.
3. Anne Mitchell
Anne Mitchell was the Executive Director of CIELAP for many years, now retired and doing independent freelancing. She's also a neighbour that I've known for a long time through Quaker connections, and last year I kept running into her on the street and chatting and thought I'd offer her a website to help her with her freelancing. I don't usually do small little sites like this, but Anne is not only extremely generous and nice, I like what she has to say. I hope you do too. As far as the Drupal goes, it's an example of what Drupal can do simply, and mostly out of the box.
4. La Leche League of Canada
This site was another refugee from a previous service provider. In this case, the provider had done a nice job of the design, but didn't know much about Drupal and even less about CiviCRM (and their CiviCRM-knowledgeable staff had then left ...). But La Leche did have a committee member who knew a lot about websites (she runs her own web consulting business), and was happy to learn about Drupal and CiviCRM, so in this case, all I did was an easy migration to my web server, and a bit of advice from time to time, and she finished off all the hard bits to get the site ready. I'm going to presume that you know about the la leche league, and if you don't, well, go visit their nice new site. I hadn't had any direct experience with them before, but I know three La Leche leaders and they're all excellent people.
Lowdown on Launching a Drupal Website
I don't usually launch quite so many sites together, so it was a great opportunity to reflect on that process. I still don't like calling it a 'launch', which I presume is an attempt to make a technical switch sound more exciting that it is. I guess for some sites, there can be a little celebration, but here's what actually went into these Drupal site launches.
The domain name system is what maps domain names to IPs (which are then assigned to computers with a complementary system). So technically, a launch is typically just a change in a dns entry, which redirects browsers from the old machine (or placeholder site) to the new one. The tricky bit here is often that the dns can be hosted with the old provider, so you may need to create a new dns service, which then requires instead a change in the domain registration, which has it's own complications. In addition, you'll also want to consider what happens to mail and other uses of that domain and any subdomains, and decide on the behaviour of the domain with and without www. This is a classic example of how what should be a simple technical job can easily ambush you into a disaster, and it's a good idea to give this process ample advance thought and time.
2. Domain name change
If you've set up the site with a temporary domain, then switching it usually has implications for Drupal and CiviCRM. If you've built it well, then it can be relatively seamless, but inevitably there are a few places where you have to change a configuration or two or three, and then you'll want to clear some caches, and make sure that anyone who visits the old/temporary domain is processed nicely. All this is made more complicated by the fact that the dns system is distributed and changes are propagated in a somewhat unpredictable way, so you have to think through a variety of scenarios of how people will be experiencing your site over the following 12 hours or so.
The only useful little trick here I use is: after I've swtiched the DNS, i edit my own hosts file to force my DNS over ahead of most people, and then I can test how it will look before most people see it using the new domain. This is especially useful when you're inheriting a site from someone else who may not have been quite as thoughtful about the launch process (to be generous), and hacked in all sorts of hard-coded links to the temporary domain. Usually you can fix most of those using grep and replace within the theme, but there are some cases where you have to dig into the module settings to fix stuff (particularly with any links that go into generated mail messages, where they have to be absolute ..). If you've got time on your hands or if links to the old domain is going to really break your site, you can also run a spider through the site (say, using wget --mirror), and then check the output there.
3. Google, etc.
It turns out that the machine view of the internet is as important or more than the human view. I say that not because I love machines, but because many human views are filtered through the machine (I don't just mean your computer, but other computers). The prime example of this is how your site will appear when someone searches for it with google: not only does the ranking matter - the title and snippet will also affect how people experience your site, and you don't want to mess that up.
In other words, stuff like SEO and analytics are important to the long term health of your site. So after I've cleared all the caches and made sure the site looks good to humans, then I go into analytics and set up the account and tell google to start paying attention to it on its new domain, and then i go into google webmaster and do the same. When google recognizes the site, then I use the drupal module xmlsite map to generate a site map and submit it. This serves two functions: I get a nice listing of all the nodes so that i can check that they've got reasonable addresses (using pathauto of course), and when I submit it to google, it means a faster and more efficient spidering of the site.
After those three pieces are done (the third sometimes has to wait a day), and any little remaining bugs are identified and squashed, then the site is truly 'launched'. After a day or two, I'll also revisit the site at the temporary domain and make sure that any visits to that domain are suitably redirected (i.e. a permanent redirect).
So, I'm now off for a couple of weeks, and when I get back I'll try and finish off all the big projects that have actually been my bread and butter for the past 4 months.
Monday, May 31, 2010
Congratulations to the organizers!
[Oct 4, 2010 update: actually, extra congratulations for bringing in an amazing line of keynote speakers, including Dries himself: keynote page. Now - go register before it fills up!]
Thursday, May 20, 2010
But once you add the complexity of users publishing their own content, or adding an 'intranet' where only some people are able to see private content, it gets tricky. Unlike Wordpress and Joomla that are very focussed on the simple website permissions model, Drupal was originally built as a platform for collaboration, and it's got some great tools for addressing all kinds of access use cases. In particular, sometime around when I started working with Drupal, the node access system of access control was born.
Unfortunately, the problem of access control is inherently challenging due to its complexity, and every time I try to implement it, I have to reread about the Drupal node access model to remember how it works. I've just had to do this again while implementing a customized Open Atrium installation for the Centre for Social Innovation, and so this posting is my attempt at writing it down to help me remember it for next time, and to share it with others.
When you step beyond the basic web publishing model, then the key question is: who can do what (view, edit, etc.) with what page? The obvious model is then to define those access permissions for each page, but after trying that once, you'll realise that you probably don't want to do that because:
a. it takes too long to set up (n nodes x m users x p permissions = n x m x p checkboxes to add).
b. it's impossibly cumbersome to maintain.
Or, put another way, we usually have a different mental model that includes some intermediate abstractions that helps us answer the question "who can do what". Actually, I'm oversimplifying already - I've been describing the problem of access to "node" pages - but what about those listing pages? You'll also want to restrict what's listed, depending on who's looking, right? Let's forget that problem for a second though.
As an example, in Drupal, users are grouped according to 'role', and nodes are grouped by 'type'. In a common scenario, you might want to restrict nodes of type "secret" to only be visible by users with the role "spy". In fact, that scenario is nicely covered by some standard modules, so you can implement that one without thinking about it too much. Another important example is in the case of "Organic Groups", where access for group nodes depends on a users' membership in and administrative permissions for the group.
Those two examples are not only the most common implementations of node access, they're also an excellent demonstration of how to understand and implement Drupal's node access system. The key to simplifying node access in Drupal is the abstraction of groups of related users and nodes. This is done with the concept of a 'realm' which is best thought of as the context of groups of users accessing a particular group of nodes. If you were just thinking of the first example, then this seems like excessive abstraction, but if you think of the Organic Groups example, you'll see why we need this, where the "group of users" abstraction is contextually dependent on the node you're looking at.
So: here's a recipe for implementing Drupal's node access system.
1. write down in english all the access control restrictions that your system needs. Give each group of users and nodes sensible names.
2. reread the example implementation in the api.drupal.org (search for node_grants and the node_access_records hooks).
3. define the node_access_records function per node: for a given node, what are the relevant 'grants', i.e. permissions for each user group, where 'realm' corresponds to the user group within the context of that node.
4. define the node_grants function, which then converts the abstraction of the realm, operation and user into the actual yes or no.
So the machine actual flow will go like this:
1. find the corresponding row(s) in the node_access table for the current node.
2. for each row, run the corresponding node_grants function with the current user and realm to determine that user's permission for that node.
Be sure to test the outcome, particularly if you're protecting important content!
And finally - let's go back to the question of listing pages. If you've been following along so far, you may wonder why we have to use all these abstractions, and in fact, you wouldn't have to. You could (and can) just intervene with the nodeapi hook and hide nodes that the current user shouldn't have access to.
But it turns out that by using the 'realm' concept, and breaking the answer to the question into these two hooks, we can very efficiently generate lists of nodes per user that respect the node access.
If it sort of makes sense now, but you think someone has already solved your particular use case, then check out this overview of node access modules.
Thursday, May 13, 2010
Short version: thanks Matthew Clarke for fixing a bunch of problems that were primarily affecting the recurring payment option.
Thursday, April 22, 2010
I went to DrupalCon last year as well, and I valued it because I learned some things that are hard to learn from just reading websites, I got to know some of the personalities of the project, I met some useful people to know, and I generally got re-inspired about working with Drupal. Still, I'm mostly an introvert and had decided not to go this year until I found out it was in San Francisco, and the added attraction of visiting my brother who lives in Palo Alto tipped the scales for me. So, following are my loose collections of notes from the sessions I attended.
Fusion and Skinr
This was a presentation by two of the TopNotch themers. I've used their premium themes in almost all my recent projects – it's a great way to get a nice looking theme easily and not very much cost. It seems they are now converting entirely to using Fusion and Skinr for all their themes, so it seemed like a good idea to try and understand what that means.
It turned out to be mostly understandable. Skinr is a module that enables “skinning”, which is how most applications (not just websites) choose to expose themselves to visual design changes. That means, being able to change a limited number of pieces of the design, typically colours and fonts, sometimes limited layout things. My understanding of how Skinr works is that it just injects divs around html elements, allowing the theme to apply collections of css to multiple elements.
So Fusion is the base theme that does this, by using the theme settings stuff to allow an administrative interface that is pretty point and click, where you say “apply this skin to this element” kind of thing. The goal here is to have some easy ways of doing standard things reliably and repeatably, so that's a good thing. It also implies and strategy with html/css that isn't semantic, which isn't great, but maybe isn't so bad. In any case, it looks like it's fulfilling a particular use case that is true for most of my work, so I'm pretty convinced so far.
CiviCRM for Developers
I went to a “Birds of a Feather” meeting for CiviCRM developers. It was excellent to finally meet the core CiviCRM team (minus the famous lobo), and find out what David Greenberg really looks like (younger and more handsome than you might think). The session was mostly introductions, a little confusion, and then some breakout sessions where I learned some useful things.
The Chaos Tools Suits (ctools) is a helper module created by Earl Miles (also known as Merlin of Chaos, of views and panels fame), that he created to be able to share some of his techniques that he uses in those two modules. It's actually a suite of tools that live in separate files and are loaded conditionally, so it's not a bunch of dead weight even if you're only using some of those tools. The presentation was not by Earl, but by one of the modules co-maintainers and documenters, which is a nice example of how the Drupal community works well. The module comes with an examples module to help understand the various pieces, which you should check before believing any of my notes below.
- CSS Tools. This is a handy bunch of functions that can analyse style sheet content. It can read, parse and filter stylesheets and then output style sheet content based on that, so you can use it to do super overrides. For example – you could convert all the Arial fonts on a page to Verdana...
- Dependent fields. This looks like a better way to do what I've done with custom jquery in themes before. It allows you to dynamically (i.e. via AJAX) modify a form based on the user's entries. So for example, you can modify the list of provinces available based on the selected country.
- Drop-down menu links. This is just a simple, handy theme function that allows to stick little drop-down menus (say, to little configuration icons, like gear wheels).
- Exportables. This is the engine for being able to export/import views, i.e. a way of encapsulating database settings into a php array representation that can be copy/pasted and put into revision control.
- Multistep forms. This works differently from other multistep forms in that it's tying together separate forms with state information.
Barry Jaspan is a big guy at Acquia, who's responsible for designing their systems of hosting lots of big commercial sites on Amazon's web service (AWS) intrastructure. One reason they do that is so that they can actually provision a server capable of lots of traffic, in about 20 minutes. He was talking about some of the specific issues/solutions that relate to this platform. What I found most interesting was:
- Acquia uses AWS for almost everything
- the challenges and solutions were quite understandable
- it's in use on such a big production environment (though only in the past year, how things have changed from a year ago!).
Website Development Process
During the afternoon, I was visiting the sponsors and had a nice conversation with a shop owner in Texas, who was a big fan of scrum. After that I went to a talk called “Defining and Pitching your Process”, and it made me want to read more about scrum, but reassured me that what I thought I know about the client end of the development process might still be true (or at least, shared by enough other people to pass as true).
Fields in Core
The last session I went to was by Karen Stevenson, who's responsible for the date module and maintaining cck. She was talking about the process of moving fields into core. Unfortunately, it's not looking very clean or easy, sounds like there's still a lot of little pieces to finish before the D6 → D7 migration paths are sorted out. But the abstraction of the concept of fields is pretty great – it means:
- you can attach fields not just to nodes, but to any 'entity' (user, comments, but potentially other entities).
- Field content doesn't have to be in the database, it can be stored anywhere.
I caught most of the “advanced drush” session in the overflow lounge – showing that drush is alive and well (or at least still fascinates lots of developers). There were a few presenters, and it had the feel of a “look at all the cool things drush can do” show. In any case, Drush is the way to do all things command line, and it can do a lot more than you might expect. Two of the cool features I didn't know about (maybe because they're only available in the latest 3.x version that just came out) are:
- The “shell” - you can run a little command that just defines a bunch of shell aliases, so that you're back in a shell, but with your various site configuration stuff set and all your favourite drush commands now available via unix-like commands that you can mix in with your normal shell commands.
- Aliases. One of the hassles of using drush is having to tell it the different configuration variables each time (i.e. document root and site). I currently do this by having a drushrc file in my current directory and accessing drush via a shell that automatically pulls it in, but the canonical and presumably better way is to use aliases that live in your .drush directory. I'll have to see whether this really makes my life easier ...
The last session I went to was the plenary by David Cole, a young guy who's on the new media team at the white house, and who helped get whitehouse.gov to use Drupal. He was pretty inspiring in showing that young, idealistic people can still have a remarkable impact on government, and it was a proud moment when some of the authors and maintainers of Drupal modules got credited with helping it all happen – an open source contributor's dream.
I don't like conferences, but DrupalCon is pretty worthwhile. I'd been a little jaded of late about the enterprise emphasis of Drupal as led by Acquia, and it might still derail and distract from the community-driven open source ethos of the project, but it hasn't yet. So I'm still in.
Tuesday, February 09, 2010
If you're wondering what's happening to my existing work - not to worry, I'm still providing the same service to my current clients. Like all change, there may be some hiccups, but my goal is to be able to provide a better service by focussing on what I do best, and getting help with the things that I don't do as well.
Thursday, February 04, 2010
I've been meaning to write up some notes about "the environment" and me.
I'm on the tree-hugging side of the fence politically, and have some of the old-fashioned skin-flint variety of environmentalist in my personal choices [keep the house cool, hang up my laundry to dry, walk/bike/TTC most of the time, own an old used Prius, caulk and insulate my old house obsessively, pay attention to my gas/electricty bills, etc.].
But for the purposes of this post, I want to point out that my website development business is also "environmental", even though I don't flaunt it very much.
- I work from home [no extra office to heat, no commuting].
- I host my sites on a "green" server at rackforce.ca
- By focusing on only Drupal, I can set up my server to be optimized for Drupal hosting, which is more efficient [both at a machine/energy level, as well as my time].
- I host quite a few environmental organizations.
So if you're looking for "the environmental choice" in your web development or hosting, you could do worse than using my services.