Plone Conference Barcelona 2017

It was awesome to be back at Plone Conference this year. Finally! We have had participation in Plone conferences in 2009, 2011–2012 and 2014–2017, but for me the previous one was years ago: Plone Conference Bristol in 2014. Needless to say that I have missed the warm and welcoming atmosphere of a Plone conferences, and It's my pleasure to report that Barcelona did not let me down. Even the weather was still warm there in this October.

This year there was no single big Plone news at the conference. The latest major release of Plone CMS was released already two years ago, and the next feature release is still waiting for its gold master. Yet, there was still a lot of good news, and putting all the puzzle pieces together resulted in a clear picture of the future of Plone.

Disclaimer: These are obviously just my personal opinions on all these things Plone...

Plone Conference Barcelona was so much of fun that I took a piece of it with me back home.

Plone 2020 and beyond

At first, let's say it clear that Plone CMS remains to be a safe bet for a long-term enterprise CMS solution. If there ever was any doubt, whether Plone could make it to Python 3 in-time before the end of Python 2.7 maintenance in 2020, that should be no more. Plone will make it.

All the major blockers seem to have been solved, and the rest is just hard work left for our community (check some related talks by Alexander and Hanno about the recent events on that). Python 3 version of Zope application server powering Plone is already in beta, and it is scheduled to be released within a year. Plone, for sure, has still plenty of packages to be ported from Python 2.7 to Python 3, but there are already many sprints scheduled to continue that work in near future (including the already completed Barcelona Conference sprints). We might even have an alpha version of Plone on Python 3 before end of 2018.

In addition that, it's always good to mention, that Plone Foundation has continued to do its usual great job in all the possible paper work around managing Plone's copyrights and trademarks.

All these should be good and relaxing news for any long-term Plone user.

Let's go frontend!

The greatest challenge for Plone CMS seems to be keeping up with the ever increasing UX expections of the day, while complying with the high accessibility standards. After Plone 5 rewrote the default theme and whole front-end resource management in Plone, there are no longer blockers for using any current front-end tech with Plone. But just being able to use some tech is not enough – also the real work for better UX needs to be done. And even a lot has been done for Plone 5 and 5.1, that work seems to never end.

Plone Conference Barcelona included a great amount of front-end, user experience and accessibility related talks to educate our community. So many that I can only mention a few.

At first, there were talks regarding the current Plone user interface: Johannes gave a bit technical, but very comprehensive talk how the new frontend resource registries in Plone 5 really work. My talk instructed, how to combine the ancient powers of Zope application server with the modern Plone 5 theming support to achieve shorter iterations and faster deployments when developing new UX features. Our Rikupekka talked about our migration experiences from Plone 4 to Plone 5, and gave a demo about of the UI features we have developed using the approach I discussed in my talk. Finally, I want to mention Wildcards' Kim's talk about Castle CMS, which really showcased, how much difference well lead and focused UX development for Plone based distribution could do in just about a year. Although, the fact that Castle's development had to be forked a bit from the main Plone distribution is also telling, how difficult it is to make the same UX please everyone.

Then there were many talks about the future: there's a new branch of Plone user interfaces built completely in JavaScript on top of the great Plone REST API (which Timo gave a nice presentation about). With Plone REST API it's possible to combine the great and robust CMS features of our secure Plone backend with leading edge JavaScript based frontend. It also makes Plone based solutions feasible for the current generation of frontend developers, because only very basic Plone knowledge is needed to get started. And while there is no complete replacement of Plone user interface in JavaScript yet, there are SDK like projects with many familiar UI components already for ReactJS, Angular (check Eric's talk) and even on VueJS.

If these don't feel ambitious enough, there was one more thing: Albert's talk about Pastanaga UI – a proposal for next generation UI for generic CMSs.

Guillotina – server for a more civilized age

I'm not sure how common mistake it is, but at least we have sometimes ended up using Plone as a framework for projects, for which Plone was not really the most optimal solution. That has happened, because Plone has some very unique features we love and trust: object database with URL traversal, extremely flexible Zope Component Architecture, and very proven security model especially designed for hierarchical data.

At Barcelona conference, Nathan from Onna presented their new ”AsyncIO REST Resource Application Server” called Guillotina (open sourced through Plone Foundation)r What makes Guillotina very special and interesting is that it has all those unique features we have learned to love in Plone ”framework”, but with minimal server footprint and first class support for asynchronous programming using Python 3 AsyncIO event loop. That should allow Guillotina to go places where no Plone has gone before.

I really hope the next year brings us a suitable project to try Guillotina in practice...

There and back again

To summarize all this, here's my picture of the future of Plone on the base of Plone Conference Barcelona 2017 in three sentences:

• Plone CMS as we know it remains here to stay – the current users remain safe with Plone
• Plone REST API and all the UI SDKs based on it ”save Plone” by making it a feasible solution for content management related progressive web apps
• Guillotina ”saves Plone developers” by allowing them to transfer their current Plone ”framework” knowledge into era of high-performance Python 3 AsyncIO microservices.

Obviously there was a lot more in the conference than this. There was a lot of great talks by talented speakers. It was great to see all the old friends and make some new ones. I had a chance to meet my GSOC 2017 student Oshane Bailey. And there are no parties like parties in Plone Conferences.

Thanks once again for all the organizers. It was a pleasure to be there.

We'll see if I get to see Tokyo next year...

Photo of me, Oshane Bailey and David Bain by Maik Derstappen. They said this pose is to honor Usain Bolt.

Midsummer Sprint Final Report

Plone Midsummer Sprint was held at Jyväskylä, Finland, on 3rd–7th of July 2017. The main goal for the sprint was to polish content editors' user experience on the current Plone CMS release. While a lot was accomplished for that original goal, the final sprint topics adapted to follow the interests of its volunteering participants.

The sprint was sponsored by the University of Jyväskylä. In addition, the sprint was accepted to receive strategic funding from Plone Foundation to cover the possible travel and accommodation costs of Albert Casado, the designer of Plone 5 user interface.

This is the final report of Midsummer Sprint.

Participants

There was 18 participants at Midsummer Sprint in total. The participants included, of course, our organisers, our local development team for Plone based and related services, a GSOC student working on a Plone Foundation mentored topic, a professional UX designer and many well known names from Plone community.

Our sprint organisers:

• Asko Soukka, University of Jyväskylä
• Rikupekka Oksanen, University of Jyväskylä

Our local development team:

• Jussi Rajala, University of Jyväskylä
• Jussi Talaskivi, University of Jyväskylä
• Matti Pohjanvirta, University of Jyväskylä
• Valtteri Karppinen, University of Jyväskylä

Google Summer of Code student:

• Mikko Hursti, University of Jyväskylä

Plone community participants:

• Albert Casado, Hewlett Packard
• Alexander Loechel, Ludwig-Maximilians-Universität München
• Eric Steele
• Maik Derstappen, Derico
• Paul Roeland, Clean Clothes Campaign
• Peter Holzer, agitator Weblösungen
• Philip Bauer, Starzel.de
• Stephan Klinger, der Freitag
• Sven Strack
• Timo Stollenwerk, kitconcept GmbH
• Victor Fernández de Alba, kitconcept GmbH

Thank you for all the participants! Many of you came to Jyväskylä with members of your family, and it was our pleasure to host every one of you!

Results

According to the feedback that we got from our participants, the sprint was considered a very successful one. Maybe the sprint was unable to fix as many content editing related issues as we were hoping for, but it definitely managed to get the best out of its participants:

At first, thanks to this sprint, all members of our local Plone development team at IT Services of University of Jyväskylä got to learn how Plone is being developed. They got their contributor agreements submitted to be able to push their patches to upstream, they learned how those patches were made, tested and reviewed in practice, and they got to know many experienced Plone developers in person. Of course, they also got to feel the spirit of an international open source development sprint, from the community with probably the longest continuing tradition on it.

The same was true for Mikko Hursti, a local student attending the sprint while working on a Google Summer of Code project for Plone Foundation. This year only four students from Finland were accepted for Google Summer of Code program. Mikko's project is about implementing a drag and drop user interface for customising content listings on Plone.

During the sprint, our local development team mainly focused on confirming and fixing the issues reported for PloneFormGen on Plone 5. PloneFormGen is the most famous form builder for Plone, used by most of the content editors at our university. Also, its unrivalled flexibility really showcases many of the unique features of Plone CMS. Before the end of the sprint, Jussi Rajala and Valtteri Karppinen submitted at least four related pull requests for it and got very familiar with the large code base in question. Not bad at all for their first sprint.

• https://github.com/plone/Products.Archetypes/pull/93
• https://github.com/plone/Products.Archetypes/pull/94
• https://github.com/smcmahon/Products.PloneFormGen/pull/199
• https://github.com/smcmahon/Products.PloneFormGen/pull/200

Security is one of the most advertised features of Plone CMS. What makes it particularly great feature is that content editors don't really need to know or think about it. It just works. Maik Derstappen made spectacular work at the sprint by cleaning up and fixing the code base for a major security feature of Plone called safe HTML transform. It's a configurable HTML filter, which prevents content editors from accidentally (or with malicious objectives) introducing security issues in their edits. Maik's work guarantees that the feature really works as advertised also on Plone 5. With the new synchronised configuration, it also works better than ever with TinyMCE, the WYSIWYG HTML editor used on Plone.

• https://github.com/plone/Products.CMFPlone/pull/2098
• https://github.com/plone/Products.PortalTransforms/pull/20
• https://github.com/plone/plone.outputfilters/pull/25

The latest major Plone release, Plone 5, introduced a completely redesigned content management toolbar for all logged-in users. But not all logged-in Plone users are content editors, and when they are not, the new toolbar could a distraction. At the sprint, Peter Holzer continued on earlier work by Jens Klein and introduced Show Toolbar -permission to control when the full toolbar is shown, and an alternative member tools -dropdown menu to show a minimal set of required actions (including logout) for logged-in users when the real toolbar is not shown. The default behaviour, of course, remains unchanged.

• https://github.com/plone/plonetheme.barceloneta/pull/131
• https://github.com/plone/Products.CMFPlone/pull/2096
• https://github.com/plone/Products.CMFPlone/pull/2100
• https://github.com/plone/plone.app.upgrade/pull/127
• https://github.com/plone/plone.app.layout/pull/128

Plone 5 was also the first Plone CMS release to ship with a complete multilingual content support. With the bundled Multilingual Support -add-on enabled, Plone supports easy-to-use one-to-one translation of each content page for any language. In addition to fixing some reported issues in it, Peter Holzer and Asko Soukka redesigned its key user interface element, Translate-menu on the toolbar, to be much simpler and more consistent with the other menus in Plone's editing toolbar. With the redesigned Translate-menu soon in place, we really expect to see a good drop in translation related support requests that we currently receive from our content editors.

• https://github.com/plone/plone.app.drafts/pull/6
• https://github.com/plone/plone.app.multilingual/pull/283
• https://github.com/plone/plone.app.multilingual/pull/284
• https://github.com/plone/plone.app.multilingual/pull/285

Plone provides, and has always provided, readable and meaningful web addresses for any content stored on it. Even better, it's really easy to content around on Plone. But what happens to those old web addresses when content gets moved? Correct. Of course Plone keeps track of moved content and maintains a redirection table for them. Unfortunately, until now, the redirection table has been invisible for Plone web masters. Luckily, Stephan Klinger championed to implement the PLIP for adding the missing redirection management UI for Plone, based on the existing (but lacking) RedirectionTool add-on. While Stephan made the most of the work, Asko helped him by implementing paging and filtering for the new redirection management control panel.

• https://github.com/plone/Products.CMFPlone/commits/plip-1486-redirection

Our farthest participant was Eric Steel, the Plone release manager, who came to the sprint all the way from the United States. Eric started his sprint by releasing the latest beta of Plone 5.1 for developers, but then, together with Philip Bauer, completed the long-awaited refactoring of Plone login forms. The current login forms used in Plone 5 are relying on a form framework, which is no longer used anywhere else in Plone. Upgrading Plone login forms to use the current form frameworks in Plone has been one of many prerequisites for upgrading Plone to Python 3. It's just awesome that this has finally been done, and we cannot wait for being able to customise Plone login forms in a more sustainable way now. In addition, Philip did also fix a few known issues in the in-place content migration from Plone 4 to Plone 5.

• https://github.com/plone/Products.CMFPlone/issues/2092
• https://github.com/plone/plone.login/commits/master
• https://github.com/plone/plone.app.contenttypes/pull/413

A lots and lots of work was also done to enhance the official Plone documentation at the sprint. To really get an idea, how Plone documentation applies traditional CI practices to ensure high quality documentation, we really recommend watching the presentation that Sven Strack gave us at the sprint. Sven Strack, Paul Roeland and Alexander Loechel worked at least for better tools for building and testing the documentation, better theme for the upcoming Plone 5.1 documentation, better readability of the docs on mobile devices, refactoring robot generated screenshots (helped also by Asko), documentation on developing better documentation for contributing the docs itself and probably much more. In addition, Paul also fixed a few reported accessibility issues on Plone 5 and tested Asko's approach for building Plone themes with Webpack. Alexander used some of his sprint time to mentor his GSOC student Pavithira KC on implementing new add-ons listing for plone.org.

• https://github.com/plone/sphinxtheme.plone/commits/master
• https://github.com/plone/documentation/commits/5.1
• https://github.com/plone/Products.CMFPlone/commits/robodoc
• https://github.com/plone/Products.CMFPlone/issues/1619
• https://github.com/plone/Products.CMFPlone/pull/2089
• https://github.com/plone/plone.app.layout/pull/126
• https://github.com/plone/plonetheme.barceloneta/pull/130

Other interesting developments at the sprint included Maik implementing sub-templates into bobtemplates.plone, Alexander adding tox based test setup into bobtemplates.plone and Jussi Talaskivi enhancing plone.docker. Once Maik's enhancements for Plone add-on templates at bobtemplates.plone are available, they will allow to make significantly cleaner initial templates and provide sub-template based scaffolding throughout the add-on lifetime. Alexander's examples for tox based test setups pave the way for testing add-ons easily on both Python 2 and Python 3. Jussi clarified documentation on using Plone Docker images (also as a cluster), investigated into slimming Plone Docker images and then researched the remaining issues in installing Plone with pip, the official Python package installer.

• https://github.com/plone/bobtemplates.plone/pull/206
• https://github.com/plone/plone.docker/issues/69
• https://github.com/plone/plone.docker/issues/70

Finally, there was the dream team of Albert Casado, Victor Fernández de Alba and Timo Stollenwerk working on a new CMS user interface and user experience concept called Pastanaga UI. If something makes this project special, it's the team. Timo is expert on test driven development, and experienced Plone entrepreneur, who really knows the challenges on the current CMS market. Victor is expert on Plone as enterprise CMS. He has authored a book on using Plone for intranets, and has managed many hundreds of Plone sites during his prior position at Universitat Politècnica de Cataluny. Albert, of course, is best known for his design of the current Plone 5 default theme called Barceloneta. And the best part: they all have successfully worked together before.

At the sprint, Albert completed many of his earlier designs for Pastanaga UI components and composed a visual style guide to help with the reference implementation and later adaptations. Thanks for Alberts prior experience on Plone, Pastanaga UI simplifies many Plone key features to be easier to use than ever, especially on mobile. All this his available on request until the reference implementation has been completed. Victor and Timo supplemented Albert's work by bootstrapping a ReactJS based reference implementation of the design. Their goal is to complete Plone REST API compatible minimal viable product with the new UI soon, and attract resources for a more complete implementation later. Since the first implementations will be fully JavaScript based, it not yet sure, if Pastanaga UI would be adaptable for the current Plone UI.

• https://github.com/plone/pastanaga

Many of our sprint days started with a presentation on some of the sprinted or otherwise current topics, including a presentation on Pastanaga UI by Timo and Albert. Recordings of those presentations are now publicly available on our Plone based video publishing platform Moniviestin.

Program

While the code is important, Plone sprints are also about the people, who participate them. Plone community has a long tradition for dining or just hanging around together after those very intensive work days, and Midsummer Sprint did not make an exception. We also tried our best to take account that many of the participants travelled with their families.

Most of us dined together every day from Sunday evening to Friday evening. While many of the participants were still arriving on Sunday, we dined at Restaurant Shalimar inside the train station, to be able to welcome travelling participants and their families as they arrived.

In the Monday evening we dined at Restaurant Sohwi, which has always been popular place among the students and staff of our university due to its close proximity to our main campus areas.

On Tuesday, we wrapped up the day a bit earlier than usual and took a guided tour at Alvar Aalto museum (tour being sponsored by the university) to learn and get inspired by Aalto's design. In the evening, we walked a short forest trail at Haukanniemi together, and experienced a beautiful and warm sunset while enjoying a traditional Finnish camp fire with Finnish sausages (and surprisingly without mosquitoes).

In the Wednesday evening we dined together at viking restaurant Harald, where it was possible to enjoy many Finnish delicacies including reindeer, gravlax, Scandinavian rye and malt bread, lingonberries and cheese bread with cloudberries. We were lucky enough to catch the summer discount of the restaurant to make that evening a bit more affordable.

On Thursday, we organised a traditional Finnish sauna evening with street food at Opinkivi Sauna, a private sauna apartment owned by The Student Union of the University of Jyväskylä. Two separate dressing rooms and saunas at Opinkivi Sauna ensured that everyone was able to enjoy Finnish sauna in comfort.

And finally, in the Friday evening, we ended the sprint with the remaining participants by dining together at restaurant Trattoria Aukio, near the harbour of Jyväskylä, and then, said the goodbyes.

Thank you all, and see you soon!

Blazingly fast code reload with fork loop

The Plone Community has project long traditions for community driven development sprints, nowadays also known as "hackathowns". For new developers, sprints are the best possible places to meet and learn from the more experienced developers. And, as always, when enough openly minded developers collide, amazing new things get invented.

One of such event was the Sauna Sprint 2011 at Tampere, Finland, organized by EESTEC. During the sprint, from the idea by Mikko Ohtamaa, with the help from top Zope and Plone developers of the time, we developed a fast code reloading tool, which has significantly speeded our Plone-related development efforts ever since.

So, what was the problem then? Plone is implement in Python, which is a dynamically interpreted programming language, already requiring no compilation between change in code and getting change visible with a service restart. Yet, Plone has a huge set of features, leading to a large codebase, and a long restart time. And when you are not doing only TTD, but also want to see the effects of the code changes in running software (or re-run acceptance tests), restart time really affects development speed. Python language did had its own ways for reloading code, but there were corner cases, where everything was not really reloaded.

While our tool is strictly specific to Plone, the idea is very generic and language independent (and, to be honest, also we did borrow it from a developer community of another programming language). Our tool implemented and automated a way to split the code loading in Plone startup into two parts: The first part loads all the common framework code, and the second part only our custom application code. And by loading, we really mean loading into the process memory. Once the first part is loaded, we fork the process, and let the new forked child proces to load the second part. And every time a change is detected in code, we simply kill the child process and fork a new one with clean memory. What could possibly go wrong?

That was already almost five years ago and we are still happily using the tool. Even better, we did integrate the reloading approach for a volatile Plone development server with pre-loaded test fixture: each change to code or fixture restarts the server and reloads the fixture. acceptance tests for it. No more time wasted for the other developer to continue from where you left.

University of Jyväskylä Tech Blog - Technology, Development and Culture

Did you know that University of Jyväskylä IT Services does a lot of in-house software development?

In this blog we will tell you about what technology choices we have made and why, how we develop software, how we utilize Open Source software and how we keep up inspiring culture in our teams.

In this article I will tell shortly about who we are and what services we have. In later posts we will dig into the interesting how.

Screenshot of our build monitor

University of Jyväskylä - facts and figures

The University of Jyväskylä is a nationally and internationally significant research university and an expert on education that focuses on human and natural sciences.

• Seven faculties
• 15 000 students
• 2600 staff members
• Budget of about 211 M€/year

University of Jyväskylä resides in the middle of Finland.

Compared to many other higher education organizations in Finland we have relatively large group of people dedicated to IT - 90 people, of which

• about 30 do IT services (network, servers, email, AD etc.)
• about 40 work at IT support and software support
• about 20 work with maintaining software and developing new software, of which about 15 are developers

Our web-based, inhouse services, some examples

• Study data system Korppi (used by all students and teachers)
• Plone CMS for our websites (80 sites, over 250 000 pages of content, 2 000 000 visitors/month), e.g. https://www.jyu.fi
• Moniviestin video publishing platform - since 2003, thousands of videos
• Koppa and Avoin Koppa LMS - integrated to Korppi and plagiarism detection
• Payment systems - over 1 M€ gross/year and about 25 000 payment transactions made/year
• TUTKA research portal
• iRODS research data store
• Multiple customised web-services
• Jybar - simple yet powerful toolbar connecting our different web based services
• ROTI - study information system 
• Integrations to nation wide systems
• Migrations from legacy software to new ones
• and much more!

What's to come?

We will tell concrete examples of about

• Technology choices we have made 
• Systems we have developed and maintained with relatively little amount of people
• Automated software testing, Continuous deployment 
• Docker, Ansible, Gitlab
• Java, Python, C#, JavaScript, HTML, CSS etc
• Open Source
• Being Agile 
• How to cope with public sector bureaucracy
• How to do integrations between different systems
• Successful (and perhaps not so successful) customer use cases
• And more

Stay tuned!