buytaert.net

At DrupalCon Denver, I announced the need for a strong focus on Drupal's authoring experience in my State of Drupal presentation. During my core conversation later in the week, I announced the creation of a Drupal 7 distribution named "Spark" (formerly code-named "Phoenix"). The goal of Spark is to act as an incubator for Drupal 8 authoring experience improvements that can be tested in the field.

I hope for Spark to provide a "safe space" to prototype cutting-edge interface design and to build excellent content tools that are comparable with the experience of proprietary alternatives. While not a final list, some initial thinking around the features we want to experiment with is:

• Inline editing and drag-and-drop content layout tools ("true" WYSIWYG)
• Enhanced content creation: auto-save, save as draft and more
• Useful dashboards for content creators
• Mobile content authoring and administration support

The vision behind the Spark distribution is to be "the Pressflow of Drupal authoring experience". Pressflow provided a "spoon" of Drupal 6 with various performance enhancements that made their way into Drupal 7 core while it was in development. The same improvements were made available to Drupal 6 users so they could easily be tested in the field. With Spark, we want to test authoring experience improvements in Drupal 7 on real sites with real users and real content. We also want to target the best improvements for inclusion into Drupal 8 core.

I'm excited to announce that Acquia will fund the Spark distribution. Core developers Gábor Hojtsy and Wim Leers will work on Spark full-time starting in late May. They will work along side Angie Byron (webhchick), Alex Bronstein (effulgentsia), myself and other members at Acquia. While we have some promising candidates so far, Acquia is still seeking applicants to join the Spark team (with a strong preference to candidates located in or willing to move to the Boston area):

• Drupal UX Interaction Designer, who can conceptualize and prototype cutting-edge interface design.
• Senior Front-End Engineer, who eats, sleeps, and breaths JavaScript and jQuery (Drupal experience optional).

The Spark team will collaborate with the Drupal usability and the core development teams.

For many years now, developers around the world have celebrated and promoted the numerous benefits that open source has to offer IT and business communities. Despite the flare for technology innovation and bringing new offerings to market, the real value of the open source community is the culture of the people that represent it. A shared ethos, coupled with a collaborative working model and mutual respect has delivered and will continue to deliver cutting edge software offerings that are increasingly competing with traditional proprietary vendors.

But open source has moved beyond simply being a novelty or hobby, as its potential for huge cost reductions and delivering significant savings to the bottom line have become recognized by hard pressed businesses around the globe. Implementations of open source projects can also now be found in many countries in the government sector, with the UK, US, and France being notable examples. Only recently, it was announced that Iceland was shifting over to an open source model to help make savings and reduce the deficit.

For those of us working in the community, the only surprise with these headline-grabbing government sector implementations was that they weren’t happening faster.

When making the case for open source, despite the numerous benefits on offer, it’s vital that providers demonstrate they have the same structure and ecosystems you would expect from a major proprietary software vendor. In this context, open source offerings need to be appropriately packaged up with hosting, consultancy and the support network that many IT decision-makers consider to be a necessity for implementation. That’s why I founded Acquia, which serves as a commercial vehicle for enabling Drupal open source adoption into enterprise-size organizations, offering support and service level agreements that enterprise users expect.

But the open source community has recently seen two major developments that have fundamentally changed the perception of everything we have to offer. The first being Red Hat reaching the $1 billion USD revenue mark, which provided a huge confidence boost to open source developers that their business model is profitable and can be successful. This landmark achievement will open the floodgates to more developer-focused organizations achieving unprecedented success and puts further pressure on the traditional proprietary vendors that have dominated the IT landscape for so long.

Another landmark announcement is that Microsoft has chosen to move into the open source space, a signal of just how seriously the value of community development has become. Some expected this news to be met with a negative reaction, but the open source community should celebrate the fact that a large proprietary software organization is investing in open source and extend a warm welcome to Microsoft.

With businesses looking for IT solutions that can deliver both innovation and cost savings, there has never been a more exciting time to be involved in open source. With open source businesses reaching the $1billion dollar revenue mark and leading proprietary firms opening up new subsidiaries to invest in open source, the open source community should feel that the best days are still yet to come. Once a fast growing self-contained community, open source is now recognized as a genuine alternative to proprietary software with a serious offering that will empower businesses across the globe.

One of my favorite songs of the past 6 months. So pure.

Acquia works with many large enterprises that bet on Drupal. These organizations are doing amazing things with Drupal and innovating by breaking through prior limitations. However, in talking to our customers, we noticed that there is limited knowledge sharing and discussion happening among the heaviest Drupal users. Similar problems are often solved multiple times independently, and in incompatible ways. And since few of these large companies are vocal and active in the community, the expertise gained from solving these problems isn't making its way back into the software that all Drupal users rely on.

To help solve these issues, I'm announcing a new program called "Large Scale Drupal" as part of my group at Acquia's Office of the CTO.

Large Scale Drupal is a group of large enterprise Drupal users who meet regularly to discuss and collaborate on common problems. We provide a forum for enterprise users, listen to their needs, prioritize them as a group, and then figure out a proper way to address those needs through knowledge sharing, white papers, training and development. The intent is not to keep the outcome of these meetings just within the group. We want to share what we learn in the Large Scale Drupal group with the specific intent of it becoming a contributed project to Drupal. Once contributed, anyone is welcome to discuss and assist to the project.

So what are these projects? These are common needs for large enterprises that are considered large and complicated Drupal problems. Through a consensus-driven process our first project is working on creating a better content staging system geared toward supporting a publishing workflow. We've already started having detailed discussions and working on some of the basic architecture. We are connecting Large Scale Drupal program participants with members of the community to help advance projects like Workbench, and build new contributions like a site preview system. This program will add to those systems by helping define the needs of the users, funding some of the work, and contributing patches to the code.

The goal of these projects is to foster knowledge sharing and collaboration among members of the group and the community. The Large Scale Drupal members get the benefits of sharing their development costs with other members. The community benefits by gaining new contributions to Drupal, and an influx of expert talent into the Drupal contributor pool. Both the contributions of these companies, and the expertise that they bring to the table will help Drupal remain a long-term viable project.

I'm excited to work with the Large Scale Drupal program members to get them more involved in Drupal and become active contributors to the community. I have a big vision for Large Scale Drupal; something I hope to write more about later. For now, I wanted to announce and bring awareness to the program.

Last week at DrupalCon Denver, I gave my traditional state of Drupal presentation. A total of 3000 were present at DrupalCon. A lot of people asked me for my slides. So in good tradition, you can download a copy of my slides (PDF, 21 MB) or you can watch a video recording of my keynote.

We've already made a number of process improvements to help accelerate Drupal core's development velocity. Today, I'd like to talk about how we can better streamline the Drupal core decision-making process.

The Drupal core queue is filled with extremely passionate, intelligent and determined individuals who always strive to solve problems in the best possible way. However, we do sometimes come to disagreements about the best way to approach a particular solution, and when two or more individuals fundamentally disagree, issues can sometimes turn into stalemates; or worse, turn ugly. When this happens, it saps strength from the core development team, and makes it difficult (and scary) for new contributors to engage.

Going forward, I'd like to propose the following workflow change to help un-stick issues where opinions seem deadlocked:

1. Create a balanced issue summary at the top of the issue that reflects the current state and history of the discussion, and outlines the pros and cons of various options.
2. Move the "Assigned to" field to "Dries". (Note that access to do this is restricted to those contributors listed in MAINTAINERS.txt; you can either ask one of them to do so on your behalf.)
3. Once per week (barring travel or other things that might come up), I will go through this list and either make a decision myself, or delegate the decision to another contributor, along with a date for a decision to be made. If there is still not agreement by that date, the person delegated the responsibility will make what they feel is the best decision, and we'll move forward. If it's particularly contentious, we can tag it "revisit before release" and look at it again in November or so.

I hope that this process change will allow us to continue to make bold strides in Drupal 8, while giving dissenting opinions a chance to be heard.

It's that time of year again where we are gearing up for another great DrupalCon. Next week, 3000 Drupalists, including more than 70 Acquians, will be migrating out west to the Rocky Mountains for an action packed week filled with sessions, stickers, beer, and lots of face time with the best open source community on the planet.

There is one remarkable event that caught my attention and that speaks volumes about an important trend we're seeing: the Higher Ed Drupal Users meeting on Wednesday.

Why is this so interesting you ask? Well ... it all started with one of Acquia's team members reaching out to a couple of universities from Canada to organize a meeting at DrupalCon where they could connect and share insider tips for what works for them at their respective university. However, it turned out they were already talking on a regular basis and what they really wanted was to talk to others from universities outside of their immediate circle. Word spread, and now what began as a lunch meeting has turned into a meet-up with approximately 50 Higher Education Drupal users coming together to talk about how they can grow Drupal on their campus and overcome the common challenges they are facing. This is what DrupalCon and the Drupal community are all about!

As Drupal adoption has grown, so has adoption in Higher Education. We recently found that out of 3260 universities that we tracked, over 35% of them are using Drupal including 71 of the top 100 universities like Harvard, Duke, MIT, UPenn, Princeton, UC Berkeley, Stanford, McGill, and many more. That is pretty amazing!

But it makes sense because Drupal provides significant advantages to universities, including support for large scale mulit-site deployments, fit with centralized IT organizations, low end-user training requirements, lower costs, appeal among digital natives and young developers, support for integration with campus authentication and authorization systems, and strong content relation capabilities - particularly taxonomy support for libraries.

I look forward to meeting with this unique group of Drupal users as they learn from each other, and undoubtedly teach us more about the specific needs in higher education. DrupalCon here we come!

At the Web Services and Context Core initiative sprint earlier this month, we attempted to re-scope the initiative so that it was more manageable and less daunting. We decided to spin off one of the major components of what it was trying to tackle into its own separate initiative, the Layout initiative.

The goal of the Layout initiative is to make all elements on the page into contextual blocks that can be rearranged and organized into flexible layouts (and even layouts within layouts) through a drag and drop interface.

Specifically, the initiative breaks down as:

1. Contextual blocks: Provide the ability to pass in relevant configuration data to blocks so they can react on something other than the URL of the request.
2. Blocks everywhere: Make all page elements—from the site logo to menus to the main page content—into blocks which can be treated the same.
3. Multiple page layouts: Choose from either pre-configured layouts such as "3-Column" and "Grid" or make your own custom layouts for pages.
4. Partial page rendering: Allow for individual page components to be loaded and rendered independently, allowing for better performance and independent AJAX requests.
5. Better UI/UX to manage blocks: A visual, drag and drop interface for creating page layouts and populating with blocks.

This approach to building pages yields a number of benefits, including the ability to customize the look and feel of a site based on a variety of contextual information, the ability to render parts of the page independently for performance, increased consistency, and increased flexibility for site builders to re-use page elements in a number of contexts.

I've asked Kris "EclipseGc" Vanderwater to head up the Layout initiative team. Kris has spent a lot of time working with Panels and Page Manager, and intends to work closely with Earl Miles and other maintainers of the CTools suite. Read Kris's blog post for an overview of the initiatives' goals. Note that like all initiatives, they don't actually materialize unless people decide to help. Please join the discussions in the Blocks and Layouts Everywhere Initiative on groups.drupal.org and help work on Layout issues on drupal.org.

As the Documentation Team lead, Jennifer "jhodgdon" Hodgdon has done a fantastic job of not only keeping Drupal core's API documentation high-quality and consistent, but also of on-boarding new Drupal core contributors through the "Novice" issue queue.

Since documentation improvement patches are always welcome, and since they are unlikely to break other parts of the system, I'm happy to announce the promotion of Jennifer as a Drupal core co-maintainer for version 7 and 8. Her responsibility will be solely around documentation and code style patches, plus occasional help on "emergency" commits such as a required rollback of an accidental patch commit in order to get our automated test suite passing again.

The hope is that delegating responsibility for documentation and code style patches to Jennifer will help increase the velocity of Drupal 8 development. Not only will documentation changes go in faster, it also allows catch, webchick and myself to focus our time on bigger patches.

Welcome to the core committer team, Jennifer! :-)

Last weekend, we held a sprint at the Acquia offices for the Web Services and Context Core (WSCCI) Initiative for Drupal 8. This was an important sprint for the future of Drupal. This blog post provides a high-level overview of what was discussed and agreed upon; the different sprint participants will be laying out more technical details in follow-up blog posts.

Overall, a wide range of experience levels, skill sets, and perspectives were brought to the table, with the goal of the sprint being to clearly define the initiative’s scope, get agreement on what we wanted to accomplish and why, and lay out a clear plan for how to accomplish this.

In attendance were:

• Larry "Crell" Garfield, lead architect of the WSCCI initiative
• Daniel "sun" Kudwien, top Drupal core contributor
• Fabien "fabpot" Potencier, lead developer of the Symfony project
• Kris "EclipseGc" Vanderwater and James "neclimdul" Gilliland, who have helped develop and implement the plugin system as part of WSCCI
• Greg "heyrocker" Dunlap, initiative lead of the Configuration Management Initiative for Drupal 8
• Moshe Weitzman, long-time Drupal core contributor
• Angela "webchick" Byron, community cat herder and Drupal 7 core co-maintainer
• Randy "rfay" Fay, Justin "beejeebus" Randall and Alex "effulgentsia" Bronstein, core developers and sane voices of reason with an outside perspective
• Dries Buytaert (me), project lead

The WSCCI initiative, as envisioned by Larry Garfield, was originally set to address Drupal's web services and flexible page layout capabilities. We discovered that both would require significant changes to Drupal core, and it was difficult to build consensus online, so we decided to get together for 3 days and to flesh out what we actually wanted to accomplish, and how.

At the sprint, we first attempted to articulate all of the problems that WSCCI was trying to solve, which included: multiple page layouts, better UI/UX to manage blocks, partial page rendering (ESI, AHAH), contextual blocks, different response types per delivery callback/URL, plugin system / swappable subsystems, lazy loading bootstrap (convert subsystems to PSR-0 classes), infrastructure for building web services, dependency injection, and so on.

We then did a round of voting where we could each choose 3 of those things in order to try to determine which of those were the most important.

Two things became instantly clear during this exercise:

1. The items encompassed under WSCCI really spanned at least 3 separate major areas: Web Services, more robust ESI-based layouts (think Panels only more powerful), and cleaning up our underlying toolset to be a more loosely-coupled framework.
2. The underlying architecture to support RESTful calls to Drupal that makes all of the other things possible was deemed the most important thing to focus on.

After a good chunk of discussions, all were in agreement to scale back the scope of the initiative to just the "Web Services" piece, and spin off the Layout/blocks/related-UI parts to a separate effort.

Furthermore, some efforts, such as PSR-0 and Unified Plugin system, were only semi-related to the WSCCI initiative in the first place, and just happened to become relevant for it. Work on those efforts will continue as part of the general Framework community efforts.

Fabien was able to offer a tremendous number of insights as to how various Symfony2 components could help provide underlying structure for Drupal core to support Web Services out of the box. Essentially, most of what the WSCCI team had been trying to work toward, in the abstract, was already implemented within Symfony2. While some implementation details were different than what we had in mind, the end result is almost exactly what we have been trying to accomplish. We therefore agreed that the best way forward was to leverage several Symfony2 components within Drupal as a way to speed progress toward that end.

Some of the concrete benefits that would come out of these changes are

• An underlying framework that is a first-class REST citizen. That means developing web services becomes easier and more efficient, because the plumbing is already in place. An HTML page is a particular case of a REST service.
• Because the core system will be fully REST-ified, we'll be able to improve existing APIs and expose Drupal content in a natively RESTful way. For example, our current AJAX system doesn't comply with REST standards, but with these changes, can be cleaned up to do so.
• That in turn makes Drupal-to-Drupal communication, content staging, content sharing, and a host of other related tasks easier.
• The use of widely used libraries and techniques makes Drupal more approachable to new developers.

As it has evolved into an increasingly powerful system, Drupal has gotten increasingly complex and is not as easy to start developing with as it once was. Many developers are nervous about continuing that trend. Managing complexity is a challenge faced by many software projects, and one approach is to use standardized patterns and components.

Due to its long support for PHP 4, as well as some unique needs, Drupal does not take full advantage of standardized patterns and components. The complexity of the custom code that’s used and the non-standard architecture combines to create a barrier to entry for developers new to Drupal (both experienced and novice developers alike).

Meanwhile, the web is constantly changing around us. Web services and mobile are more important than ever, and with that comes the need to have more flexible page and layout capabilities. Now feels like the right time to modernize Drupal’s capabilities to bring it to the forefront of modern PHP systems and web systems in general.

While changing Drupal's core plumbing to address these needs, it's also a good opportunity to do so using more widely understood and modern techniques and libraries. Leveraging the work of a fellow open source community lets us get a jump on these changes to build a more powerful, more flexible, and more easily learnable system in less time than it would take to go it our own.

While these changes may seem large, and some of them are, we believe that they will achieve the right balance between empowering Drupal's design and architecture, and moving toward more modern, standard, well-tested code and techniques to empower a new generation of developers. I hope you are as excited as we are!

Last summer, I blogged about how I think about Drupal release date planning, tying it to the Gartner hype cycle and the corollary Drupal mood cycle.

The release timeline I laid out in my previous blog post was a Drupal 8 release 18 months after Drupal 7 had achieved the “Plateau of Productivity", the point in time where developing in Drupal 6 seems mostly pointless due to the maturity of Drupal 7.

At that time, I said that I felt that Drupal 7’s “plateau of productivity" was about 6-9 months away. Today, almost 9 months later, I think that by any reasonable measure we are currently there. There are over 300,000 live Drupal 7 installations, which represents nearly 50% of all reported Drupal sites. The top Drupal modules all have Drupal 7 releases, the vast majority of which are either stable releases or release candidates.

Having reached the “plateau of productivity" also means that I feel comfortable announcing the Drupal 8 release timeline. Without further ado, here is how the rest of the Drupal release cycle breaks down:

Timeline:

• December 1, 2012: Feature freeze. No new features are allowed (unless specifically exempted), focus turns instead to API and UI clean-ups and polishing of existing features.
• February 1, 2013: Code freeze: focus on bug fixes, stabilization. No API changes, instead focusing on bug fixing, preparing for release, and getting the count of critical bugs down to 0.
• August, 2013 (DrupalCon Europe 2013): Drupal 8 released, to wild, international fanfare. :-)

This means that Drupal 8 is 18 months away. Time to shift Drupal 8 core development into higher gear!

The ~6-month window for bug fixes laid out here is obviously much shorter than the 18-month window for bug fixes we ended up having with Drupal 7, but the hope is that the issue count thresholds that we’ve introduced this release will ensure this process is much shorter than in Drupal 7, since we’ll be going from approximately 15 down to 0, rather than approximately 300 to 0.

This timeline also means that if there are Drupal 8 initiatives you’d like to see happen, or other specific features or things you want to see fixed in Drupal core, now is the time to make those things happen. If you’ve never helped with Drupal core development before and would like to, stop by IRC during Core office hours, or join us at DrupalCon Denver. There will also be plenty of other sprints at DrupalCon around various Drupal core initiatives, and you can always start your own!

See you in Denver and in the issue queue! :-)

We're proud to present a new design for the Mollom.com website.

We first launched the Mollom.com site in 2007. For more than four years, Mollom.com was using the same design. As we grew Mollom, we wanted to address some of the issues that we've been stewing over since our original design. We have been planning to redesign the site for over a year now but work on the Mollom web service and developing new Mollom products have always had a higher priority so we haven't found the time to complete the new design until now.

The old Mollom.com design that we used from 2007 to early 2012.

The new design is the first step in our plans to reorganize the website. We still have updates to make to the content of some pages, for example. Already, we think the new design is a fresh new change that improves usability.

Take a look at the new mollom.com, we hope you like it!

The new Mollom.com website design.