Security advisories: Drupal core – Critical – Arbitrary PHP code execution – SA-CORE-2020-013

Arbitrary PHP code execution
The Drupal project uses the PEAR Archive_Tar library. The PEAR Archive_Tar library has released a security update that impacts Drupal. For more information please see:

Multiple vulnerabilities are possible if Drupal is configured to allow .tar, .tar.gz, .bz2 or .tlz file uploads and processes them.

To mitigate this issue, prevent untrusted users from uploading .tar, .tar.gz, .bz2 or .tlz files.

This is a different issue than SA-CORE-2019-12, similar configuration changes may mitigate the problem until you are able to patch.


Install the latest version:

Versions of Drupal 8 prior to 8.8.x are end-of-life and do not receive security coverage.

According to the regular security release window schedule, November 25th would not typically be a core security window. However, this release is necessary because there are known exploits for one of core’s dependencies and some configurations of Drupal are vulnerable.

Reported By: 
Fixed By: 

drunomics: Nuxt.js – The frontend framework for decoupled Drupal with Custom Elements

In my first blog post, I introduced the Custom Elements module: A solution for soft-decoupled Drupal! Following that, I want to put more emphasize on the frontend side of the stack in this blog post:

Selecting a frontend framework

With the backend producing custom elements markup, we need a frontend framework that is able to work with that. While there are many frameworks that play nice with custom elements, we want to pick a well-established solution with a large user base. That way, it’s more likely that the framework stays relevant and well maintained on the long term. Also, finding more integrated libraries, learning materials or developers knowing the framework becomes much easier.

According to The State of JavaScript 2019: Front End Frameworks the top 3 frameworks developers have used are React, Vue.js and Angular:

  • Angular has been not liked much by developers that used it and won’t be used again by the majority of them (see survey). Besides that, it’s provided as a whole MVC framework with lots of things potentially not needed here. So not the best fit.
  • React has a very strong user base, but does not play that well with custom elements. While there are solutions like react-slot, they are not as common and well maintained. Finally, personally I don’t like the “Everything is Javascript” approach so much.
  • Vue.js on the other hand, comes with template parsing built-in that is able to render custom elements data easily. Like React, it utilizes a virtual DOM and is well adopted and continuously growing. Finally, the Vue.js single file components are following a template based, web-standard oriented approach.
  • Since web components build upon custom elements, they seem to be the perfect fit. However, they are not, since they are not well adopted and server-side rendering web components is not a well solved issue (yet?).


Thus, Vue.js turns out to be the ideal candidate for our decoupled Drupal stack built upon custom elements. Moreover, I like it for the following:

  • It’s approachable and easy to use. It has great documentation and sticks to web standard oriented notations.
  • It comes with a set of incrementable adoptable companion libraries for further needs, like routing or state management.
  • It’s fast! It only weighs 20kB gzipped and scores well in rendering benchmarks.
  • Just like Drupal, it’s a community backed open-source project, not depending on a single large corporation. Check out the Vue.js team and its sponsors.


Once one decides for Vue.js, one quickly stumbles over Nuxt.js – the intuitive Vue framework. It comes with a set of useful conventions – inspirated by Next.js – that make it very easy and enjoyable to get started with development. It provides all necessary setup for Javascript transpilation, CSS processing and improves performance by handling route-based automatic code-spliting or link prefetching. Thankfully, it’s following Vue.js principles, thus it emphasizes the ease of use and provides great documentation.

Finally, it’s built upon a powerful, modular architecture and allows providing features as re-usable Nuxt.js modules – what makes it a great addition to Drupal. While the number of modules is nowhere comparable to Drupal, there are many useful modules available, like PWA generation, Google Analytics or Tag manager integration, or the usual frontend/CSS framework integrations with Bootstrap, Material UI or Tailwind CSS. Check out the module directory at for more.

Nuxt.js deployment options

One of the nice features of Nuxt.js is that it’s really easy to deploy your project using various ways, all from the same code-base – just by a different deployment configuration. The options are:

  • Static generation
    Generate a static site and leverage the Jamstack approach for dynamic functionality and content previews.
  • Server Side Rendering
    Render the markup using a Node.js server or Serverless functions.
  • Single Page Application

    Just serve a pre-built Javascript application to users and let the user’s browser take over rendering.

That way, one can choose the best suiting deployment target per project. Small sites without a large number of pages could be very easily statically generated and benefit from fast, cheap and secure hosting; e.g. via providers like Netlify, Amazon S3 or Github pages. If SEO is not required, running as a Single Page application does away the need for re-building after content changes and allows for highly dynamic, user-specific content and app-like UIs.

More demanding projects, requiring SEO and having a large number of content, can use server side rendering with any Node.js compatible hosting provider like or Heroku. Alternative options would be specialized hosting providers like Vercel, AWS amplify or deploying via the Serverless framework to various serverless cloud providers.

The frontproxy approach

Finally, I’d like to mention we also developed a custom, more advanced approach that becomes possible with custom elements markup: The lupus-frontproxy is a small PHP-based script that serves content as custom element markup combined with a pre-generated app shell (site header and footer). That way, large, content-heavy sites can easily run without a node.js server driving the frontend, while still providing decent SEO based upon the custom element markup delivered (Google just ignores custom elements and indexes the contained markup). However, with the rise of easy and affordable hosting options for server side rendering, such a custom built, uncommon solution seems unnecessary and not really worth the efforts.

Summing up

Nuxt.js is a great framework that makes it really easy to build a Vue.js based frontend application that renders the custom elements markup provided by the Drupal backend. Since each custom element is mapped to a Vue.js component, it’s the perfect fit for building up component-based frontends, while keeping the editorial controls in the backend.

Thanks to its flexible deployments options, we can leverage static generation for smaller sites and use server-side rendering for larger, more content-heavy sites.

Following up

Since there are so many details more to talk about, I’ll be following up with further blog posts in the coming weeks, covering the following topics:

  • Authentication & related architecture variants. Custom Elements markup & json formats.
  • A developer introduction: Creating components, adding Custom Element processors, the relationship to Drupal’s render API, Custom routes and optimizing cache metadata.
  • Handling blocks & layout builder, content previews, forms, caching & performance optimizations.

Finally, I’m going to talk more about this stack at the Drupalcon Europe 2020 in my session “Custom Elements: An alternate Render API for decoupled Drupal” at December 08 – 09:15 – so mark your calendars!





OpenSense Labs: Mistakes to avoid on your drupal website

Mistakes to avoid on your drupal website
Gurpreet Kaur
Wed, 11/25/2020 – 13:20

Building websites that are completely mistake proof is often considered to be a massive undertaking, which many-many times is not properly executed. Since there are so many parameters to fulfil and so many corners to oversee, mistakes tend to happen. You may not even realise that you have done something wrong in the development process, it could be much much later when you actually undergo a website audit that you come across the mistake that has been made.

Drupal websites are equally prone to mistakes. Despite the CMS being one of the best, there are still occurrences when things go wrong and the impact is felt on the engagement, conversions and consequently, the sales.

To not let that happen, I have compiled a list of mistakes that you need to steer clear of when building or working on a Drupal website. These are errors and oversights that many developers and content authors have experienced first-hand and you can certainly try to learn from their mistakes.

So here are the most common mistakes witnessed on Drupal websites.
A pencil is shown after having erased an error on the extreme left and the mistakes to avoid in Drupal are written in bullets in the rest of the space on a white background.

Where can you go wrong with the content? 

A good website is often considered to be the one with outstanding content, since that is what keeps the audience engaged and leads to conversion. Therefore, content is crucial for a website, both for the front-end and the back-end, so content should be one of the priorities in the website designing process. Due to this fact, there are a number of areas where the developers can go wrong with the content. 

The first common mistake witnesses with the architecture of content is using too many content types that you actually do not use. The unused content types are just going to burden your database and I am certain, you would not want an additional table in your database for three content types that you do not even use. Having content types with no nodes will have the same effect. Performing an inventory will help you get the mistake resolved.

Moving on from the unused to the used, content structures are extremely valuable for your editors who are going to fill them up and if they end up confused, what will be the point of it all? Standardising your content types is going to help you a great deal. 

  • Strike off the content types that are similar to each other, like news and articles;
  • Do not add new fields for every content type;
  • And most importantly, plan a structure prior to it, winging it may not work with your website content.

Content types have an effect on the performance of your website as well. So, if you do not want to drain the performance of your site by adding unnecessary complexity, you would be wise to avoid these mistakes.

What about your display mismanagement?

After content comes its display and Drupal is best in that game. With many different features available for use, Drupal can make your display game strong as well, you capitalise it wisely.

Creating a view for every list is both impractical and a waste of time. Focus on reusing Views as much as possible along with parameter based rendering.

Do you use PHP code in your database? If so, avoid doing that, instead you must write all the codes in the modules or themes itself.

Planning, optimisation and segregation are essentially the building blocks of a great website display. 

  • Planning to render the content types you need;
  • Optimising the Views you already have;
  • And segregating logic from presentation.

These three would have a visible effect on the display architecture.

What aspects of functionality can make your site lag behind?

The functionality of a Drupal site depends on the number of modules used and the way they interact with each other. Your code and how much of it you use is a major contributor in that. 

The most common mistake in this sense is the ignorance of code standards. This becomes more of a problem when there are more than one developers and everyone is using a different type of code. In such a situation, not only would the standard be lost,  but it would also become difficult for a developer to understand the other’s code. Therefore, the adherence to Drupal’s Coding Standards becomes a great help to uniformalise the code and make the functionality a breeze. 

Another obstacle in functionality are unorganised patches. Code modifications and bug fixes mandates the implementation of patches, however, they become a problem whenever there is an update. You can forget all about re-apply the patch or forget to change it in accordance with the new version. This can very well affect the functionality of your website, so organising them is essential. 

Having too many modules, too many roles and too much custom code, despite there being contrib modules for the same is bound to affect the functionality as well. Evaluate and re-evaluate your site for its needs to overcome these functionality hindrances.

Is your performance and scalability not up to the bar?

User Experience is directly proportional to the performance of your website; the more streamlined the performance is, the richer would the UX be. 

Here are three scenarios that can impact your performance in all the wrong ways.

  • The foremost is improper JS/CSS aggregation settings. This is supposed to combine and compress JS and CSS files from the modules in the HTML, leading to lesser load times and higher performance. And you will be saying goodbye to that with the improper aggregation.
  • The next mistake is that of inundating your site with too many modules. Drupal may have numerous modules to offer, but you can’t be using too many of them. Doing so would only slow you down and hamper your security as well. Only keep the modules that you would be using, messing up your code, performance, overheads and security is simply not worth it.
  • A sound cache strategy also goes a long way in performance enhancement. Caching too soon, caching at lower levels and not knowing what and when to cache all contribute in a lowered performance.

Drupal websites can be scaled by millions of users within seconds and that is what makes these sites great. Drupal offers many modules to enhance the performance and scalability, Blazy, Content Delivery Network and Server Scaling, being just a few of them. Not installing these could be deemed as a mistake.

Are you facing possible security breaches?

Security has become one of the major concerns amongst website builders. Therefore, protecting your business from the menace of hackers and all the havoc they can cause is paramount. 

You must have your security measures in place, however, there still may be certain areas where you may have become complacent and that just gives the break the hackers need. 

  • Primarily, you need to keep your website updated, all the core and contrib modules, despite you using or not using them. Updating a module would mean that Drupal’s security protocols are being updated with them and you make yourself secure with that. You cannot have your projects falling behind on various levels of Drupal’s security advisories.
  • Now, you can install the “ Update Manager” module to keep yourself updated. The “Available Updates” will give you a friendly reminder of applying the available security updates.
  • Next on the list of security blunders is not giving the Input Filters in Drupal their due importance. You might have configured the full HTML Input Format to every user or you might have completely disabled the HTML filtering. Both of these instances can give malicious code to enter your website and you know what happens then.
  • Continuing on similar lines, many sites also configure their servers improperly leading to unwanted access to them. On some occasions, servers are seen displaying their version numbers, which is like giving an open invitation to hackers. Server configuration and permissions should be a priority for every site builder.
  • It is also important to ensure that all the users accessing your site by logging into it are the ones you want. By implementing a password policy, removing old users and updating staff roles, you will be taking a step towards better security.
  • User roles are quite important in running a website, however, they can become overused quite quickly too, which not only slows down your website, but if they are misconfigured, it can lead to major security breaches.

Drupal has proven to be one of the best CMSs in terms of its security measures, it has you covered from every corner, but only if you let it. From granting secure access to providing granular user access control along with database encryption and preventing malicious data entry, Drupal will keep your site protected, provided you let it.

Have you made any infrastructural oversights?

The infrastructure of your website is decided by the stacks you have, which includes the server, database and the software layers like Varnish. Going into development with a firm plan for your infrastructure is the only way to go, an oversight in this area can be quite damaging. 

The common mistakes in this area are;

  • The size of the website’s stack is extremely large or extremely small.
  • Not preparing for growth by consistently checking the logs for error and the identification of the weaklings.
  • Having an ideal sized server, but not configuring it properly, which can make the traffic forego Varnish.
  • Allowing remote connections to the server can make the website more vulnerable.

Misconfiguration can be avoided by simply using the right tools for it. MySQLTuner is one amongst many, its performance suggestions help in improving the infrastructure as well.

Are you following your maintenance goals?

Maintenance of a website starts after the development is done and continues for the entirety of the life of the website. Considering this fact, you have to be very diligent with the maintenance process as making the wrong moves can be brutal.

Here are some of these wrong moves.

  • Not following Drupal updates is a more common mistake than you would think. By doing this, you are going to be hampering security and making your site vulnerable to Drupalgeddon attacks.
  • On the contrary, there are also times when we update the modules, but we forget to remove the older versions. This too happens a lot of the time and can cause many problems.
  • It is not just the modules that need to be updated, the development environment should also be up-to-date and friendly for testing.
  • Then there is the code, which is not using the Version Control System like Git, even the deployment of files should come directly from there. Also, using it, but not leaving messages for other developers related to the changes made can lead to chaos. It is, thereby important to always keep the VCS repository clean.

The crucial aspects of maintenance is time and consistency. When you do it timely, only then would it become a worthy practice. The review and assessment of your architecture in timely intervals along with all the logs, be it Apache or Drupal is a great headstart for the maintenance process.

Are you universally accessible?

Websites today need to transcend the parameters that used to confine them and their audience in the past. The sites and applications need to be built on a foundation that would make them fit for each and every user. Drupal has worked for the same, it aims to make websites accessible to all, including people with disabilities, by making itself an all-accessible tool. 

Web accessibility has become of the essence today, and persons with disabilities are at the core of it. Websites need to be designed keeping in mind their needs, be it a broken hand or a temporary sight loss. It isn’t just me who believes this, World Wide Web Consortium’s guidelines agree with me as well. W3C two sets of guidelines are ardently followed by Drupal and your website should do the same, thus, support and foster inclusion. 

Despite its importance, many developers tend to overlook the accessibility features and that is where they go so very wrong. 

  • Not focusing on balanced contrast levels that can work under sunlight;
  • Not incorporating a form validation error verbiage to aid the visually impaired; 
  • Not using buttons over CTAs.

These may seem like minor errors to you, but they can go a long way in making your Drupal site accessible to everyone.

Is your site SEO friendly?

Being SEO friendly is almost as important as building a website. Search engines bring in big numbers of traffic, so optimising them is crucial; yet we tend to forget to fine-tune the SEO details and focus on all the other aspects. At the end of the day, a website is an online business and business cannot survive without its clients and being SEO friendly is the way to go. Going wrong in this area can be extremely detrimental.

Look at the following aspects of a website and see how many you are and aren’t doing. 

Are your URLs user friendly?
Are your images small in size, with filled out alt texts?
Are you making your paragraphs short to make the text easy to scan through?
Are you using robots.txt for pages that you do not want crawled?
Are you creating an XML roadmap to help Google easily understand the structure of your website?
Are you researching your keywords?
Are you adding internal links to make your less popular pages gain attention through the more popular ones?

A positive answer to all of these means that your SEO game is strong and a contrary answer would let you know your mistakes.

To avoid the contrary from happening, Drupal provides a number of modules to help you capitalise on the SEO front. The SEO checklist module is a proof of that as it helps you by ensuring that you are following through on the latest SEO practices. Then there are the modules that aid your URL precision, like Redirect, Pathauo and Easy Breadcrumbs.From easing the process tags to helping in communication with search engines to providing the essentials for editing, Drupal has all the right SEO modules in its corner and not using these would be a colossal mistake on your part. Read our blog, The Ultimate Drupal SEO Guide 2020, to know more about these. 

Can being multilingual pose a problem for you? 

Today, languages that are regionally spoken have started getting more prominence than ever before, especially in the international community. A french website would not be successful in India, if it is in French, not many people speak that language, so it would have to be in a locally accepted language. Being multilingual also opens the doors for many mistakes to occur. 

  • Using the same URL for all of your multilingual websites; 
  • Not giving the user a chance to avoid a redirect to the international website;
  • Using an automated translator, instead of actually hiring content authors fluent in the language;
  • Foregoing to translate the imbedded parts of the site like meta tags and descriptions;
  • Not focusing on the foreign market trends and the keywords appropriate to it;
  • And lastly, not writing the content in accordance with the local language and dialects. You can’t be calling ice lollies popsicles sticks in India.

You have to be totally attuned with the language of the region that you have followed for the multilingual project to work.

Is having a multisite presence worth it?

Depending on your business and its needs, having multiple sites can be a good solution for you. However, managing then can become a bit of a hassle and often lead to big blunders. 

Some examples of such blunders are;

  • Traffic is one of the major concerns here. Running multiple sites means you have one codebase and many sites on it, so if one is inundated with traffic, all of them could slow down as a result.
  • A mistake in the syntax of one site could mean a mistake in the syntax of all.
  • Updates become a headache as well. For Drupal sites, you have to run an update.php in order to update the site and doing that on multiple sites is going to bring on the headache.
  • And finally, if you do not use Aegir, you are going to regret going multisite.

Is your Decoupled Drupal approach the right one?

Drupal offers an impressive front-end technology to make your presentation layer as good as you want, yet it does not include all the front end technologies there are on the market. Taking advantage of JavaScript and Static Site Generator would mean to decouple Drupal and separating the front-end from it. Even if you want to take on decoupling, it may not want to take on. The decoupled Drupal can bring more drawbacks then. 

  • If you wish to capitalise Drupal’s in-built features, Decoupling would be a mistake, since you would end up parting with them.
  • If your front-end requirements and Drupal’s front-end capabilities are aligned, taking on Decoupling would only be an unnecessary effort on your part.
  • If you do not have either the budget or resources to tap into the hottest technologies, then even if you want them it is not going to be fruitful.
  • If you are only publishing content at one place, you would have no need for decoupling.

For a detailed explanation, read our blog, When to Move From Monolithic to Decoupled Drupal Architecture.

Finally, what about web hosting, are you doing it the right way?

Web hosting services that provide your website its own space on the internet are pretty common. There are far too many web hosts to count, yet the decision to choose one is not easy at all, since there are too many considerations to keep in mind. 

Some of the common mistakes to avoid which signing on a web host are;

  • Testing web hosts is not uncommon, it is the right way to know whether they are valuable. However, testing on your website that is primarily bringing in the traffic could be unwise, especially if you are using a free service. Therefore, not registering with a different party can be colossal.
  • Another mistake is trusting too easily without knowing the host for too long. Therefore, not partnering with one that has a long trial could be a mistake. The longer the trial period on offer is, the more reliable the host is going to be.
  • Taking on a web host is a huge commitment, so you have to be sure that you are in the good. Not doing your due diligence before the commitment is not the right way, comparing the pricing and features along with checking if they have blacklisted IPs.
  • Not tracking your hosting uptime and speed can also be a problem. Also not checking what guarantees for uptime are provided by the hosts for the same would not be wise. If there is a lapse between the guaranteed and actual uptime, keeping a track would give you the opportunity to ask for compensation.
  • Lastly, you cannot afford to not have a backup of your site and that too regularly. You will only have the recent version of your files and assets, if you back them up.

The Bottom Line 

Every aspect of your website is important, consequently, you have to be mindful of them all. If you are not, mistakes will happen and they will cost you your site’s performance, its security and your potential customers and sales. In order to keep that from happening, you have to avoid all of the aforementioned mistakes and ensure that your website is impeccably built and maintained on all platforms. 

blog banner
A child has his face covered with his hands and two question marks are on both his sites.

blog image
A pencil is shown with an eraser tip after haviing erased a mistake.

Blog Type
Is it a good read ?

Promet Source: Connection and Humor in a Year of Unexpected Challenges

Thanksgiving this year will be different from any we’ve ever experienced, but then again, the same could be said for pretty much every aspect of 2020.

At Promet Source, we attract talent from all over North America and the world, so we had a bit of a head start navigating remote work requirements. We were still mindful though, every day, of the many ways that Covid-19 was having an impact on our teams and our clients. 

Acro Media: Drupal 8 to Drupal 9: The Easiest Upgrade in a Decade | Acro Media

Do you know the version number of the browser you’re using? What about the Operating System version you’re using? Chances are, you have no idea what the current version you’re using of any major software. And that’s the way it should be.

Tips for upgrading to Drupal 9

  • Make sure your version of Drupal is up-to-date.
  • Use Upgrade Status to see if your website is ready for upgrading
  • If you have custom code, use Drupal Rector to identify code that needs to be updated

Throughout the software industry, there is a movement to more frequent, easier, updates. The reason behind more frequent updates is that everyone tends to keep software up to date and more secure, the easier and more frequent those updates come. Soon, you may not know the major or minor version of your website software, just that it is up-to-date and secure, which is the ultimate goal of any update, upgrade or version release.

What version of Drupal am I running?

Chances are if you’re using Drupal, you are using version 7 or 8. In June 2020, version 9 of Drupal was released and the last minor version of Drupal 8 was released. Both version 8 and 9 contain the same features and code. The only difference is that Drupal 9, or D9 as it is referred to in the industry, removed deprecated code and updated third-party dependencies.

(Source: – How to prepare for Drupal 9)

The image above shows the timeline for Drupal 8 and its version releases since 2019. The upgrade cycle in version 8 for minor releases was established to be a single release roughly twice a year. Now that Drupal 9 has been released, there will be an end-date for support for Drupal 8, but that is not scheduled until November 2, 2021. In fact, the upgrade from 8 to 9 is so painless, version 10 will be released in 2022, likely to even less fanfare, as it will also be the same as the most recent version 9.

Upgrading between all minor versions of Drupal, including the jump to version 9 is advised, but is a much simpler process than version upgrades have been in the past. See what has to say about it here. However, if your website was recently created or released since 2016, it’s likely that you’re on Drupal 8, and the upgrade should be extremely straightforward and relatively painless.

If you find yourself on version 7 of Drupal, you can absolutely upgrade straight to D9 and skip D8 altogether. The rebuild would likely take the form of an entirely new website, but the benefits of going right to D9 are two-fold: avoiding end of life issues for D8 in 2022 and jumping on a platform that will enable to you go headless, adopt the best media, personalization, translation, and content creation tool that open source has to offer.

Why migrate to Drupal 9?

Running end-of-life platforms come with a set of problems that over time, will end up costing your company time and money. When official support for D7 runs out in 2022 (the same time as D8), security updates, bug fixes and creation of new modules for those versions will also go away. This opens your system to being more vulnerable to cyber attacks, potential downtime and a lack of up-to-date features that your customers would expect from your web presence.

Jumping right into a new build with D9, you benefit from having the long-term official support from the Drupal community including security releases that help protect your website from various vulnerabilities. D9 also removes outdated code and uses the new version of PHP, which is important in terms of security.

Other reasons to upgrade to D9, from

  • Easiest upgrade in a decade, and a commitment to easy upgrades in the future. Never replatform again.
  • Continuous innovation, cutting-edge new features, reaching new digital channels while strengthening our foundation in content management.
  • Dramatically easier tools, intuitive solutions that empower business users to take advantage of the power of Drupal.

Currently, we are on the cusp of the first minor release of Drupal 9, which is planned before the end of the year. Most large ecosystem modules in Drupal have complete support for Drupal 9, including Drupal Commerce version 2.

Tips for Upgrading to Drupal 9

  • Make sure your version of Drupal is up-to-date.
  • Use Upgrade Status to see if your website is ready for upgrading (this module will flag modules that need to be upgraded to Drupal 9).
  • If you have custom code, you can use Drupal Rector to identify code that needs to be updated and, in many cases, can be automatically upgraded.

Still not sure that the upgrade to Drupal 9 is right for your organization? Have questions about the best way to handle upgrading your Drupal site? Our team is here to help you answer those kinds of questions. Check out our Drupal and Drupal Commerce Development page for more details on our services, or contact our experts to get started on your Drupal upgrade today.

Josh has been creating websites since 2000, has been a Drupal community member since 2008, and is currently the Team Lead of the Investment Division at Acro Media. He spends most of his free time playing board games and yelling at his cat with his wife.

Specbee: How to Migrate to Drupal 8 from a SQL source in 6 Simple Steps

How to Migrate to Drupal 8 from a SQL source in 6 Simple Steps
Akshay Devadiga
24 Nov, 2020

Much has been spoken about the importance and benefits of migrating to Drupal 8. Drupal migration is the most significant part of a development workflow. We need to ensure that content gets transferred seamlessly without losing out on any critical user information and data. Check this complete guide for a successful Drupal 7 to Drupal 8 migration.  

There are several ways to migrate to Drupal 8 using various sources. We have already written about how to migrate from CSV to Drupal 8. Other sources include SQL, XML, JSON, etc. In this blog, we will be discussing about migrating to Drupal 8 using SQL as a source.

Why SQL?

While choosing the data source largely depends on the source of the existing data required to be migrated, some of the other common reasons for choosing SQL source for a Drupal 8 migration are:  

  • It is easy to write queries to get the required data by connecting to the database.
  • Migrating the data from one server to another will be quicker than any other method.
  • Reduces the usage of many contributed modules.
  • No need of views data export which is used for exporting views CSV data from Drupal 7 sites.
  • We will not need the Views password field which is used for migrating sensitive information (passwords in hash), since we are using DB query.
  • Migrate source CSV module is also not needed since we are using SQL source.

Let the Migration Process begin!

In this blog, we will be migrating users to a Drupal 8 site. Following are the steps we will take to migrate to Drupal 8
1.    Create a custom module for the migration.
2.    Reference the source database.
3.    Define the migration YML and map the identified Drupal fields.
4.    Create a source plugin for migration.
5.    Process single value, multi-value and address fields.
6.    Run migration using the drush command line tool.

Step 1: Create a custom module for the Drupal 8 migration.

First, let’s create a custom module as you create in Drupal 8. Check this detailed blog for creating custom modules in Drupal 8. Here we are creating a module called “company employee migrate”. The module structure is as follows:
custom : Consists of basic information regarding the module and its dependencies.

company_employee_migrate.install : This will be used for writing PHP scripts that should trigger while installing and uninstalling the module. In our case, we are deleting the migration configuration when the module is uninstalled. See the below screenshot for the script.
Source databasecompany_employee_migrate.module : This will be used for defining the general hooks for the site. These are the initial things needed for the module. We will be explaining the remaining in the next steps.

Step 2: Reference the source database 

Next, we need to set up the source database from where we are extracting the data. Update the settings.php file for your Drupal site by moving to webroot -> sites -> default – > settings.php

Add the new database connection below the default connection as shown in the below screenshot. “migrate” is key for the source database. 
Drupal Module

Step 3: Define the migration YML and map the Drupal fields.

Now, we need to identify the fields which we want to migrate and map those fields in the migration yml. In this example we are migrating User Id, User Name, Password, Email, Status, Created Timestamp, First Name, Last Name, Job Role, Mailing Address etc.

After the identification of fields, we need to define the migration at migrate_plus.migration.company_employee.yml. Now let’s have a closer look at migration yml and their mappings.
Migration YML and Mapid: Unique id for the migration yml.

label: Display name for the migration.

migration_group: Migration group name.

source:  Annotation Name of the migration source plugin. This will be defined in the @MigrateSource annotation in src/Plugin/migrate/source/CompanyEmployee.php.

destination: Name of migration destination plugin. In this case, it’s entity:user since we are migrating user entity.

process: In process, we will be mapping the Drupal fields with the source fields. The left-side values are field machine names and the right-side values are the field names which we pass from the source plugin.

migration_dependencies: This is an optional key. This will be used only if there is any dependency on other migrations.

Step 4: Create a source plugin for migration

Migration source plugin is the heart of the SQL migration. Below is the detailed explanation for the source plugin. 

  • The source plugin can be created at src/Plugin/migrate/source/CompanyEmployee.php 
  • The namespace will be Drupalcompany_employee_migratePluginmigratesource.
  • The @MigrateSource annotations include the source plugin id that we map in the migration definition.
  • Here we are extending the Abstract SqlBase class provided by the core migrate module.

See the below screenshot for reference.
source plugin for migrationThe source plugin must implement the following three methods –

query() : This will return the source fields data by connecting to the source database. See the below screenshot which will return the field data where the user id is greater than 0 and user mail is ending with “”.
source plugin for migration
fields() : This method will return the available fields in the source. Below, I have added the code snippet for the list of available fields along with the baseFields().
basefieldsgetIds() : This method will return a unique ID for the source row. The below code will return a user ID of type integer since uid will be unique for each user.
Drupal ModuleApart from these above-mentioned methods we also have:

prepareRow() : This method will be called once for each row. We are loading the data here from different tables and processes according to the requirement. Any property we create using $row->setSourceProperty will be available in the process step. We will be using the Drupal 8 DB query to get the data in prepareRow().

baseFields() : This will contain an array of basic fields from “users_field_data” that can be used by the query() method. Find the code in the below image.
drupal 8 migrate 

Step 5: Process single value, multi-value and address fields.

In Drupal, we have different types of fields and processing some fields can get a little tricky for developers while migrating content. I have added the code snippets for some fields below:

Single value fields : These fields include text fields, boolean fields, email, etc. Find the code snippet below for getting a single value field; here the field_first_name can be set as below.
Process single value
Multi Value fields: These fields include user roles, job roles, multi value select fields, checkboxes, etc. For a multi value field we need to return an associative array. Find below the code snippet for the “roles” field.
Address Fields: Migrating the address field provided by the address module will be little different from migrating other fields. We need to preprocess the queried data into an associative array with the proper keys as shown below for field_mailing_address.
Now all set to run the migration by installing the company_employee_migrate module in the Drupal 8 site.

Step 6: Run the migration using drush command line tool

Finally, we are ready for our SQL migration. To run the migration, we need to install drush (if you are not already using it).

List of useful drush commands for Drupal migration:

$drush migrate-status : This will return the status of migration with details.
migrtion using drush$drush migrate-import migration_id : This will import the data from source to the site.
drush migration$drush migrate-reset-status migration_id : If the execution of the script has stopped or paused, the migration status will display as “Importing”. This command will reset the migration status to “Idle” so that we can proceed with the migration import.
drush migration$drush migrate-rollback migration_id : This will be used for rolling back to its previous state.
drush migration

Shefali ShettyApr 05, 2017


Lucius Digital: Update OpenLucius | Major facelift for this social productivity Drupal distro

Update OpenLucius | Major facelift for this social productivity Drupal distro
Last months we had more than 300 people testing the alpha-3 version of OpenLucius: a social productivity platform -build into a Drupal distro. We interviewed them and soon came to the conclusion that the base layouts needed big improvements. It was received as ‘mhew..’, we agreed. So we went to work and released alpha-4 today. We implemented a complete new base theme from scratch: clean, lean, fast and Bootstrap 4 based. Goal is to leave all the room for custom branding and other design needs.