November 30, 2020

WebKitGTK and WPE WebKit Security Advisory WSA-2020-0009

by The WebKitGTK Project

  • Date Reported: November 30, 2020

  • Advisory ID: WSA-2020-0009

  • CVE identifiers: CVE-2020-13543.

Several vulnerabilities were discovered in WebKitGTK and WPE WebKit.

  • CVE-2020-13543
    • Versions affected: WebKitGTK before 2.30.3 and WPE WebKit before 2.30.3.
    • Credit to Marcin ‘Icewall’ Noga of Cisco Talos.
    • Impact: Processing maliciously crafted web content may lead to arbitrary code execution. Description: An use after free issue was addressed with improved memory management.

We recommend updating to the latest stable versions of WebKitGTK and WPE WebKit. It is the best way to ensure that you are running safe versions of WebKit. Please check our websites for information about the latest stable releases.

Further information about WebKitGTK and WPE WebKit security advisories can be found at: https://webkitgtk.org/security.html or https://wpewebkit.org/security/.

November 30, 2020 12:00 AM



November 29, 2020

Accurate Conclusions from Bogus Data: Methodological Issues in “Collaboration in the open-source arena: The WebKit case”

by Michael Catanzaro

Nearly five years ago, when I was in grad school, I stumbled across the paper Collaboration in the open-source arena: The WebKit case when trying to figure out what I would do for a course project in network theory (i.e. graph theory, not computer networking; I’ll use the words “graph” and “network” interchangeably). The paper evaluates collaboration networks, which are graphs where collaborators are represented by nodes and relationships between collaborators are represented by edges. Our professor had used collaboration networks as examples during lecture, so it seemed at least mildly relevant to our class, and I wound up writing a critique on this paper for the class project. In this paper, the authors construct collaboration networks for WebKit by examining the project’s changelog files to define relationships between developers. They perform “community detection” to visually group developers who work closely together into separate clusters in the graphs. Then, the authors use those graphs to arrive at various conclusions about WebKit (e.g. “[e]ven if Samsung and Apple are involved in expensive patent wars in the courts and stopped collaborating on hardware components, their contributions remained strong and central within the WebKit open source project,” regarding the period from 2008 to 2013).

At the time, I contacted the authors to let them know about some serious problems I found with their work. Then I left the paper sitting in a short-term to-do pile on my desk, where it has been sitting since Obama was president, waiting for me to finally write this blog post. Unfortunately, nearly five years later, the authors’ email addresses no longer work, which is not very surprising after so long — since I’m no longer a student, the email I originally used to contact them doesn’t work anymore either — so I was unable to contact them again to let them know that I was finally going to publish this blog post. Anyway, suffice to say that the conclusions of the paper were all correct; however, the networks used to arrive at those conclusions suffered from three different mistakes, each of which was, on its own, serious enough to invalidate the entire work.

So if the analysis of the networks was bogus, how did the authors arrive at correct conclusions anyway? The answer is confirmation bias. The study was performed by visually looking at networks and then coming to non-rigorous conclusions about the networks, and by researching the WebKit community to learn what is going on with the major companies involved in the project. The authors arrived at correct conclusions because they did a good job at the later, then saw what they wanted to see in the graphs.

I don’t want to be too harsh on the authors of this paper, though, because they decided to publish their raw data and methodology on the internet. They even published the python scripts they used to convert WebKit changelogs into collaboration graphs. Had they not done so, there is no way I would have noticed the third (and most important) mistake that I’ll discuss below, and I wouldn’t have been able to confirm my suspicions about the second mistake. You would not be reading this right now, and likely nobody would ever have realized the problems with the paper. The authors of most scientific papers are not nearly so transparent: many researchers today consider their source code and raw data to be either proprietary secrets to be guarded, or simply not important enough to merit publication. The authors of this paper deserve to be commended, not penalized, for their openness. Mistakes are normal in research papers, and open data is by far the best way for us to be able to detect mistakes when they happen.

Collaboration network
A collaboration network from the paper. The paper reports that this network represents collaboration between September 2008 (when Google began contributing to WebKit) and February 2011 (the departure of Nokia from the project). Because the authors posted their data online, I noticed that this was a mistake in the paper: the graph actually represents the period between February 2011 and July 2012. The paper’s analysis of this graph is therefore questionable, but note this was only a minor mistake compared to the three major mistakes that impact this network. Note the suspiciously-high number of unaffiliated (“Other”) contributors in a corporate-dominated project.

The rest of this blog post is a simplified version of my original school paper from 2016. I’ve removed maybe half the original content, including some flowery academic language and unnecessary references to class material (e.g. “community detection was performed using fast modularity maximization to generate an alternate visualization of the network,” good for high scores on class papers, not so good for blog posts). But rewriting everything to be informal takes a long time, and I want to finish this today so it’s not still on my desk five more years from now, so the rest of this blog post is still going to be much more formal than normal. Oh well. Tone shift now!

We (“we” means “I”) examine various methodological issues discovered by analyzing the paper. The first section discusses the effects on the collaboration network of choosing a poor definition of collaboration. The second section discusses a major source of error in detecting the company affiliation of many contributors. The third section describes a serious mistake in the data collection process. Each of these issues is quite severe, and any one alone calls into question the validity of the entire study. It must be noted that such issues are not necessarily unique to this paper, and must be kept in mind for all future studies that utilize collaboration networks.

Mistake #1: Poorly-defined Collaboration

The precise definition used to model collaboration has tremendous impact on the usefulness of the resultant collaboration network. Many collaboration networks are built using definitions of collaboration that are self-evidently useful, where there is little doubt that edges in the network represent real-world collaboration. The paper adopts an approach to building collaboration networks where developers are represented by nodes, and an edge exists between two nodes if the corresponding developers modified the same source code file during the time period under consideration for the construction of the network. However, it is not clear that this definition of collaboration is actually useful. Consider that it is a regular occurrence for developers who do not know each other and may have never communicated to modify the same files. Consider also that modifying a common file does not necessarily reflect any shared interest in a particular portion of the software project. For instance, a file might be modified when making an interface change in another file, or when fixing a build error occurring on a particular platform. Such occurrences are, in fact, extremely common in the WebKit project. Additionally, consider that there exist particular source code files that are unusually central to the project, and must be modified more frequently than other files. It is highly likely that almost all developers will at one point or another make some change in such a file, and therefore be connected via a collaboration edge to all other developers who have ever modified that file. (My original critique shows a screenshot of the revision history of WebPageProxy.cpp, to demonstrate that the developers modifying this file were working on unrelated projects.)

It is true, as assumed by the paper, that particular developers work on different portions of the WebKit source code, and collaborate more with particular other developers. For instance, developers who work for the same company typically, though not always, collaborate most with other developers from that same company. However, the paper’s naive definition of collaboration should ensure that most developers will be considered to have collaborated equally with most other developers, regardless of the actual degree of collaboration. For instance, consider developers A and B who regularly collaborate on a particular source file. Now, developer C, who works on a platform that does not use this file and would not ordinarily need to modify it, makes a change to some cross-platform interface in another file that requires updating this file. Developer C is now considered to have collaborated with developers A and B on this file! Clearly, this is not a desirable result, as developers A and B have collaborated far more on the development of the file. Moreover, consider that an edge exists between two developers in the collaboration network if they have ever both modified any file anywhere in WebKit during the time period under review; then we can expect to form a network that is almost complete (a “full” graph where edges exists between most nodes). It is evident that some method of weighting collaboration between different contributors would be desirable, as the unweighted collaboration network does not seem useful.

One might argue that the networks presented in the paper clearly show developers exist in subcommunities on the peripheries of the network, that the network is clearly not complete, and that therefore this definition of collaboration sufficed, at least to some extent. However, this is only due to another methodological error in the study. Mistake #3, discussed later, explains how the study managed to produce collaboration networks with noticeable subcommunities despite these issues.

We note that the authors chose this same definition of collaboration in their more recent work on OpenStack, so there exist multiple studies using this same flawed definition of collaboration. We speculate that this definition of collaboration is unlikely to be more suitable for OpenStack or for other software projects than it is for WebKit. The software engineering research community must explore alternative models of collaboration when undertaking future studies of software development collaboration networks in order to more accurately reflect collaboration.

Mistake #2: Misdetected Contributor Affiliation

One difficulty when building collaboration networks is the need to correctly match each contributor with the correct company affiliation. Although many free software projects are dominated by unaffiliated contributors, others, like WebKit, are primarily developed by paid contributors. Looking at the number of times a particular email domain appears in WebKit changelog entries made during 2015, most contributors commit using corporate emails, but many developers commit to WebKit using personal email accounts, such as Gmail accounts; additionally, many developers use generic webkit.org email aliases, which were previously available to active WebKit contributors. These developers may or may not be affiliated with companies that contribute to the project. Use of personal email addresses is a source of inaccuracy when constructing collaboration networks, as it results in an undercount of corporate contributions.  We can expect this issue has led to serious inaccuracies in the reported collaboration networks.

This substantial source of error is neither mentioned nor accounted for; all contributors using such email accounts were therefore miscategorized as unaffiliated. However, the authors clearly recognized this issue, as it has been accounted for in their more recent work covering OpenStack by cross-referencing email addresses from git revision history with a database containing corporate affiliations maintained by the OpenStack Foundation. Unfortunately, no such effort was made for the WebKit data set.

The WebKit project was previously dominated by contributors with chromium.org email domains. This domain is equivalent to webkit.org in that it can be used by contributors to the Chromium project regardless of corporate affiliation; however, most contributors with Chromium emails are actually Google employees. The high use of Chromium emails by Google employees appears to have led to a dramatic — by roughly an entire order of magnitude — undercount of Google’s contributors to the WebKit project, as only contributors with google.com emails were considered to be Google employees. The vast majority of Google employees used chromium.org emails, and so were counted as unaffiliated developers. This explains the extraordinarily high number of unaffiliated developers in the networks presented by the paper, despite the fact that WebKit development is, in reality, dominated by corporate contributors.

Mistake #3: Missing Most Changelog Data

The paper incorrectly claims to have gathered its data from both WebKit’s Subversion revision history and from its changelog files. We must draw a distinction between changelog entries and Subversion revision history. Changelog entries are inserted into changelog files that are committed into the Subversion repository; they are completely separate from the Subversion history. Each subproject within the WebKit project has its own set of changelog files used to record changes under the corresponding directory.

In fact, the paper processed only the changelog files. This was actually a good choice, as WebKit’s changelog files are much more accurate than the Subversion history, for two reasons. Firstly, it is easy for a contributor to change the email address entered into a changelog file, e.g. after a change in company affiliation. However, it is difficult to change the email address used to commit to Subversion, as this requires requesting a new Subversion account from the Subversion administrator; accordingly, contributors are more likely to use older email addresses, lacking accurate company affiliation, in Subversion revisions than in changelog files.  Secondly, many Subversion revisions are not directly committed by contributors, but rather are actually committed by the commit queue bot, which runs various tests before committing the revision. Subversion revisions are also, more rarely, committed by a completely different contributor than the patch author. In both cases, the proper contributor’s name will appear in only the changelog file, and not the Subversion data. Some developers are dramatically more likely to use the commit queue than others. Various other reviews of WebKit contribution history that examine data from Subversion history rather than from changelog files are flawed for this reason. Fortunately, by relying on changelog files rather than Subversion metadata, the authors avoid this problem.

Unfortunately, a serious error was made in processing the changelog data. WebKit has many different sets of changelog files, stored in various project subdirectories (JavaScriptCore, WebCore, WebKit, etc.), as well as toplevel changelogs stored in the root directory of the project. Regrettably, the authors were unaware of the changelogs in subdirectories, and based their analysis only on the toplevel changelogs, which contain only changes that occurred in subdirectories that lack their own changelog files. In practice, this inadvertently restricted the scope of the analysis to a very small minority of changes, primarily to build system files, manual tests, and the WebKit website. That is, the reported collaboration networks do not reflect collaboration on any actual source code files. All source code files are contained in subdirectories with their own changelog files, and therefore no source code files were actually considered in the analysis of collaboration on source code changes.

We speculate that the analysis’s focus on build system files likely exaggerates the effects of clustering in the network, as different companies used different build systems and thus were less likely to edit the build systems used by other companies, and that an analysis based on the correct data would display less of a clustering effect. Certainly, there would be dramatically more edges in the already-dense networks, because an edge exists between two developers if there exists any one file in WebKit that both developers have modified. Omitting all of the source code files from the analysis therefore dramatically reduces the likelihood of edges existing between nodes in the network.

Conclusion

We found that the original study was impacted by an unsuitable definition of collaboration used to build the collaboration networks, severe errors in counting contributor affiliation (including the classification of most Google employees as unaffiliated developers), and the omission of almost all the required data from the analysis, including all data on modifications to source code files. The authors constructed and studied essentially meaningless networks. Nevertheless, the authors were able to derive many accurate conclusions about the WebKit project from their inaccurate collaboration networks. Such conclusions illustrate the dangers of seeking to find particular meanings or explanations through visual inspection of collaboration networks. Researchers must work forwards from the collaboration networks to arrive at their conclusions, rather than backwards by attempting to match the networks to conclusions gained from prior knowledge.

Original Report

Wow, OK, you actually read this far? Since this blog post criticizes an academic paper, and since this blog post does not include various tables and examples that support my arguments, I’ve attached my original analysis in full. It is a boring, student-quality grad school project written with the objective of scoring the highest-possible grade in a class rather than for clarity, and you probably don’t want to look at it unless you are investigating the paper in detail. (If you download that, note that I no longer work for Igalia, and the paper was not authorized by Igalia either; I used my company email to disclose my affiliation and maybe impress my professor a little.) Phew, now I can finally remove this from my desk!

by Michael Catanzaro at November 29, 2020 11:46 PM



Catching up on WebKit GStreamer WebAudio backends maintenance

by Philippe Normand

Over the past few months the WebKit development team has been working on modernizing support for the WebAudio specification. This post highlights some of the changes that were recently merged, focusing on the GStreamer ports.

My fellow WebKit colleague, Chris Dumez, has been very active lately, updating the WebAudio implementation …

by Philippe Normand at November 29, 2020 12:45 PM



November 26, 2020

Notes on using Emacs (LSP/ccls) for WebKit

by Víctor Jáquez

I used to regard myself as an austere programmer in terms of tooling: Emacs —with a plain configuration— and grep. This approach forces you to understand all the elements involved in a project.

Some time ago I have to code in Rust, so I needed to learn the language as fast as possible. I looked for packages in MELPA that could help me to be productive quickly. Obviously, I installed rust-mode, but I also found racer for auto-completion. I tried it out. It was messy to setup and unstable, but it helped me to code while learning. When I felt comfortable with the base code, I uninstalled it.

This year I returned to work on WebKit. The last time I contributed to it was around five years ago, but now in a different area (still in the multimedia stack). WebKit is huge, and because of C++, I found gtags rather limited. Out of curiosity I looked for something similar to racer but for C++. And I spent a while digging on it.

The solution consists in the integration of three MELPA packages:

  • lsp-mode: a client for Language Server Protocol for Emacs.
  • company-mode: a text completion framework.
  • ccls: A C/C++ language server. Besides emacs-ccls adds more functionality to lsp-mode.

(I known, there’s a simpler alternative to lsp-mode, but I haven’t tried it yet).

First we might explain what’s LSP. It stands for Language Server Protocol, defined with JSON-RPC messages, between the editor and the language server. It was orginally developed by Microsoft for Visual Studio, which purpose is to support auto-completion, finding symbol’s definition, to show early error markers, etc., inside the editor. Therefore, lsp-mode is an Emacs mode that communicates with different language servers in LSP and operates in Emacs accordingly.

In order to support the auto-completion use-case lsp-mode uses the company-mode. This Emacs mode is capable to create a floating context menu where the editing cursor is placed.

The third part of the puzzle is, of course, the language server. There’s a language servers for different programming languages. For C & C++ there are two servers: clangd and ccls. The former uses Clang compiler, the last can use either Clang, GCC or MSVC. Along this text ccls will be used for reasons exposed later. In between, emacs-ccls leverages and extends the support of ccls in lsp-mode, though it’s not mandatory.

In short, the basic .emacs configuration, using use-package, would have these lines:


(use-package company
  :diminish
  :config (global-company-mode 1))

(use-package lsp-mode
  :diminish "L"
  :init (setq lsp-keymap-prefix "C-l"
              lsp-enable-file-watchers nil
              lsp-enable-on-type-formatting nil
              lsp-enable-snippet nil)
  :hook (c-mode-common . lsp-deferred)
  :commands (lsp lsp-deferred))

(use-package ccls
  :init (setq ccls-sem-highlight-method 'font-lock)
  :hook ((c-mode c++-mode objc-mode) . (lambda () (require 'ccls) (lsp-deferred))))

The snippet first configures company-mode. It is enabled globally because, normally, it is a nice feature to have, even in non-coding buffers, such as this very one, for writing a blog post in markdown format. Diminish mode hides or abbreviates the mode description in the Emacs’ mode line.

Later comes lsp-mode. It’s big and aims to do a lot of things, basically we have to tell it to disable certain features, such as file watcher, something not viable in massive projects as WebKit; as I don’t use snippet (generic text templates), I also disable it; and finally, lsp-mode tries to format the code at typing, I don’t know how the code style is figured out, but in my experience, it’s always detected wrong, so I disabled it too. Finally, lsp-mode is launched when a text uses the c-mode-common, shared by c++-mode too. lsp-mode is launched deferred, meaning it’ll startup until the buffer is visible; this is important since we might want to delay ccls session creation until the buffer’s .dir-locals.el file is processed, where it is configured for the specific project.

And lastly, ccls-mode configuration, hooked until c-mode or c++-mode are loaded up in a deferred fashion (already explained).

It’s important to understand how ccls works in order to integrate it in our workflow of a specific project, since it might need to be configured using Emacs’ per-directory local variales.

We are living in a post-Makefile world (almost), proof of that is ccls, which instead of a makefile, it uses a compilation database, a record of the compile options used to build the files in a project. It’s commonly described in JSON and it’s generated automatically by build systems such as meson or cmake, and later consumed by ninja or ccls to execute the compilation. Bear in mind that ccls uses a cache, which can eat a couple gigabytes of disk.

Now, let’s review the concrete details of using these features with WebKit. Let me assume that WebKit local repository is cloned in ~/WebKit.

As you may know, the cool way to compile WebKit is with flatpak. Flatpak adds an indirection in the compilation process, since it’s done in an isolated environment, above the native system. As a consequence, ccls has to be the one inside the Flatpak environment. In ~/.local/bin/webkit-ccls:

#!/bin/sh
set -eu
cd $HOME/WebKit/
exec Tools/Scripts/webkit-flatpak -c ccls "$@"

Basically the scripts calls ccls inside flatpak, which is available in the SDK. And this is why ccls instead of clang, since clang is not provided.

By default ccls assumes the compilation database is in the project’s root directory, but in our case, it’s not, thus it is required to configure the database directory for our WebKit setup. For it, as we already said, a .dir-locals.el file is used.


((c-mode
  (indent-tabs-mode . nil)
  (c-basic-offset . 4))
 (c++-mode
  (indent-tabs-mode . nil)
  (c-basic-offset . 4))
 (java-mode
  (indent-tabs-mode . nil)
  (c-basic-offset . 4))
 (change-log-mode
  (indent-tabs-mode . nil))
 (nil
  (fill-column . 100)
  (ccls-executable . "/home/vjaquez/.local/bin/webkit-ccls")
  (ccls-initialization-options . (:compilationDatabaseDirectory "/app/webkit/WebKitBuild/Release"
                                  :cache (:directory ".ccls-cache")))
  (compile-command . "build-webkit --gtk --debug")))

As you can notice, ccls-execute is defined here, though it’s not a safe local variable. Also the ccls-initialization-options, which is a safe local variable. It is important to notice that the compilation database directory is a path inside flatpak, and always use the Release path. I don’t understand why, but Debug path didn’t work for me. This mean that WebKit should be compiled as Release frequently, even if we only use Debug type for coding (as you may see in my compile-command).

Update: Now we can explain why it’s important to configure lsp-mode as deferred: to avoid connections to ccls before processing the .dir-locals.el file.

And that’s all. Now I have early programming errors detection, auto-completion, and so on. I hope you find these notes helpful.

Update: Sadly, because of flatpak indirection, symbols’ definition finding won’t work because the file paths stored in ccls cache are relative to flatpak’s file system. For that I still rely on global and its Emacs mode.

by vjaquez at November 26, 2020 04:20 PM



November 23, 2020

WebKitGTK and WPE WebKit Security Advisory WSA-2020-0008

by The WebKitGTK Project

Several vulnerabilities were discovered in WebKitGTK and WPE WebKit.

  • CVE-2020-13584
    • Versions affected: WebKitGTK before 2.30.3 and WPE WebKit before 2.30.3.
    • Credit to Marcin ‘Icewall’ Noga of Cisco Talos.
    • Impact: Processing maliciously crafted web content may lead to arbitrary code execution. Description: An use after free issue was addressed with improved memory management.
  • CVE-2020-9948
    • Versions affected: WebKitGTK before 2.30.0 and WPE WebKit before 2.30.0.
    • Credit to Brendan Draper (@6r3nd4n) working with Trend Micro Zero Day Initiative.
    • Impact: Processing maliciously crafted web content may lead to arbitrary code execution. Description: A type confusion issue was addressed with improved memory handling.
  • CVE-2020-9951
    • Versions affected: WebKitGTK before 2.30.0 and WPE WebKit before 2.30.0.
    • Credit to Marcin ‘Icewall’ Noga of Cisco Talos.
    • Impact: Processing maliciously crafted web content may lead to arbitrary code execution. Description: An use after free issue was addressed with improved memory management.
  • CVE-2020-9952
    • Versions affected: WebKitGTK before 2.28.3 and WPE WebKit before 2.28.3.
    • Credit to Ryan Pickren (ryanpickren.com).
    • Impact: Processing maliciously crafted web content may lead to a cross site scripting attack. Description: An input validation issue was addressed with improved input validation.
  • CVE-2020-9983
    • Versions affected: WebKitGTK before 2.30.3 and WPE WebKit before 2.30.3.
    • Credit to zhunki.
    • Impact: Processing maliciously crafted web content may lead to code execution. Description: An out-of-bounds write issue was addressed with improved bounds checking.

We recommend updating to the latest stable versions of WebKitGTK and WPE WebKit. It is the best way to ensure that you are running safe versions of WebKit. Please check our websites for information about the latest stable releases.

Further information about WebKitGTK and WPE WebKit security advisories can be found at: https://webkitgtk.org/security.html or https://wpewebkit.org/security/.

November 23, 2020 12:00 AM



November 20, 2020

WebKitGTK 2.30.3 released!

by The WebKitGTK Project

This is a bug fix release in the stable 2.30 series.

What’s new in the WebKitGTK 2.30.3 release?

  • Add new build option USE(64KB_PAGE_BLOCK).
  • Fix backdrop filters with rounded borders.
  • Fix scrolling iframes when async scrolling is enabled.
  • Allow applications to handle drag and drop on the web view again.
  • Update Outlook user agent quirk.
  • Fix the build with video support disabled.
  • Fix several crashes and rendering issues.

Thanks to all the contributors who made possible this release.

November 20, 2020 12:00 AM



October 29, 2020

Thu 2020/Oct/29

by Claudio Saavedra

In this line of work, we all stumble at least once upon a problem that turns out to be extremely elusive and very tricky to narrow down and solve. If we're lucky, we might have everything at our disposal to diagnose the problem but sometimes that's not the case – and in embedded development it's often not the case. Add to the mix proprietary drivers, lack of debugging symbols, a bug that's very hard to reproduce under a controlled environment, and weeks in partial confinement due to a pandemic and what you have is better described as a very long lucid nightmare. Thankfully, even the worst of nightmares end when morning comes, even if sometimes morning might be several days away. And when the fix to the problem is in an inimaginable place, the story is definitely one worth telling.

The problem

It all started with one of Igalia's customers deploying a WPE WebKit-based browser in their embedded devices. Their CI infrastructure had detected a problem caused when the browser was tasked with creating a new webview (in layman terms, you can imagine that to be the same as opening a new tab in your browser). Occasionally, this view would never load, causing ongoing tests to fail. For some reason, the test failure had a reproducibility of ~75% in the CI environment, but during manual testing it would occur with less than a 1% of probability. For reasons that are beyond the scope of this post, the CI infrastructure was not reachable in a way that would allow to have access to running processes in order to diagnose the problem more easily. So with only logs at hand and less than a 1/100 chances of reproducing the bug myself, I set to debug this problem locally.

Diagnosis

The first that became evident was that, whenever this bug would occur, the WebKit feature known as web extension (an application-specific loadable module that is used to allow the program to have access to the internals of a web page, as well to enable customizable communication with the process where the page contents are loaded – the web process) wouldn't work. The browser would be forever waiting that the web extension loads, and since that wouldn't happen, the expected page wouldn't load. The first place to look into then is the web process and to try to understand what is preventing the web extension from loading. Enter here, our good friend GDB, with less than spectacular results thanks to stripped libraries.

#0  0x7500ab9c in poll () from target:/lib/libc.so.6
#1  0x73c08c0c in ?? () from target:/usr/lib/libEGL.so.1
#2  0x73c08d2c in ?? () from target:/usr/lib/libEGL.so.1
#3  0x73c08e0c in ?? () from target:/usr/lib/libEGL.so.1
#4  0x73bold6a8 in ?? () from target:/usr/lib/libEGL.so.1
#5  0x75f84208 in ?? () from target:/usr/lib/libWPEWebKit-1.0.so.2
#6  0x75fa0b7e in ?? () from target:/usr/lib/libWPEWebKit-1.0.so.2
#7  0x7561eda2 in ?? () from target:/usr/lib/libWPEWebKit-1.0.so.2
#8  0x755a176a in ?? () from target:/usr/lib/libWPEWebKit-1.0.so.2
#9  0x753cd842 in ?? () from target:/usr/lib/libWPEWebKit-1.0.so.2
#10 0x75451660 in ?? () from target:/usr/lib/libWPEWebKit-1.0.so.2
#11 0x75452882 in ?? () from target:/usr/lib/libWPEWebKit-1.0.so.2
#12 0x75452fa8 in ?? () from target:/usr/lib/libWPEWebKit-1.0.so.2
#13 0x76b1de62 in ?? () from target:/usr/lib/libWPEWebKit-1.0.so.2
#14 0x76b5a970 in ?? () from target:/usr/lib/libWPEWebKit-1.0.so.2
#15 0x74bee44c in g_main_context_dispatch () from target:/usr/lib/libglib-2.0.so.0
#16 0x74bee808 in ?? () from target:/usr/lib/libglib-2.0.so.0
#17 0x74beeba8 in g_main_loop_run () from target:/usr/lib/libglib-2.0.so.0
#18 0x76b5b11c in ?? () from target:/usr/lib/libWPEWebKit-1.0.so.2
#19 0x75622338 in ?? () from target:/usr/lib/libWPEWebKit-1.0.so.2
#20 0x74f59b58 in __libc_start_main () from target:/lib/libc.so.6
#21 0x0045d8d0 in _start ()

From all threads in the web process, after much tinkering around it slowly became clear that one of the places to look into is that poll() call. I will spare you the details related to what other threads were doing, suffice to say that whenever the browser would hit the bug, there was a similar stacktrace in one thread, going through libEGL to a call to poll() on top of the stack, that would never return. Unfortunately, a stripped EGL driver coming from a proprietary graphics vendor was a bit of a showstopper, as it was the inability to have proper debugging symbols running inside the device (did you know that a non-stripped WebKit library binary with debugging symbols can easily get GDB and your device out of memory?). The best one could do to improve that was to use the gcore feature in GDB, and extract a core from the device for post-mortem analysis. But for some reason, such a stacktrace wouldn't give anything interesting below the poll() call to understand what's being polled here. Did I say this was tricky?

What polls?

Because WebKit is a multiprocess web engine, having system calls that signal, read, and write in sockets communicating with other processes is an everyday thing. Not knowing what a poll() call is doing and who is it that it's trying to listen to, not very good. Because the call is happening under the EGL library, one can presume that it's graphics related, but there are still different possibilities, so trying to find out what is this polling is a good idea.

A trick I learned while debugging this is that, in absence of debugging symbols that would give a straightforward look into variables and parameters, one can examine the CPU registers and try to figure out from them what the parameters to function calls are. Let's do that with poll(). First, its signature.

int poll(struct pollfd *fds, nfds_t nfds, int timeout);

Now, let's examine the registers.

(gdb) f 0
#0  0x7500ab9c in poll () from target:/lib/libc.so.6
(gdb) info registers
r0             0x7ea55e58	2124766808
r1             0x1	1
r2             0x64	100
r3             0x0	0
r4             0x0	0

Registers r0, r1, and r2 contain poll()'s three parameters. Because r1 is 1, we know that there is only one file descriptor being polled. fds is a pointer to an array with one element then. Where is that first element? Well, right there, in the memory pointed to directly by r0. What does struct pollfd look like?

struct pollfd {
  int   fd;         /* file descriptor */
  short events;     /* requested events */
  short revents;    /* returned events */
};

What we are interested in here is the contents of fd, the file descriptor that is being polled. Memory alignment is again in our side, we don't need any pointer arithmetic here. We can inspect directly the register r0 and find out what the value of fd is.

(gdb) print *0x7ea55e58
$3 = 8

So we now know that the EGL library is polling the file descriptor with an identifier of 8. But where is this file descriptor coming from? What is on the other end? The /proc file system can be helpful here.

# pidof WPEWebProcess
1944 1196
# ls -lh /proc/1944/fd/8
lrwx------    1 x x      64 Oct 22 13:59 /proc/1944/fd/8 -> socket:[32166]

So we have a socket. What else can we find out about it? Turns out, not much without the unix_diag kernel module, which was not available in our device. But we are slowly getting closer. Time to call another good friend.

Where GDB fails, printf() triumphs

Something I have learned from many years working with a project as large as WebKit, is that debugging symbols can be very difficult to work with. To begin with, it takes ages to build WebKit with them. When cross-compiling, it's even worse. And then, very often the target device doesn't even have enough memory to load the symbols when debugging. So they can be pretty useless. It's then when just using fprintf() and logging useful information can simplify things. Since we know that it's at some point during initialization of the web process that we end up stuck, and we also know that we're polling a file descriptor, let's find some early calls in the code of the web process and add some fprintf() calls with a bit of information, specially in those that might have something to do with EGL. What can we find out now?

Oct 19 10:13:27.700335 WPEWebProcess[92]: Starting
Oct 19 10:13:27.720575 WPEWebProcess[92]: Initializing WebProcess platform.
Oct 19 10:13:27.727850 WPEWebProcess[92]: wpe_loader_init() done.
Oct 19 10:13:27.729054 WPEWebProcess[92]: Initializing PlatformDisplayLibWPE (hostFD: 8).
Oct 19 10:13:27.730166 WPEWebProcess[92]: egl backend created.
Oct 19 10:13:27.741556 WPEWebProcess[92]: got native display.
Oct 19 10:13:27.742565 WPEWebProcess[92]: initializeEGLDisplay() starting.

Two interesting findings from the fprintf()-powered logging here: first, it seems that file descriptor 8 is one known to libwpe (the general-purpose library that powers the WPE WebKit port). Second, that the last EGL API call right before the web process hangs on poll() is a call to eglInitialize(). fprintf(), thanks for your service.

Number 8

We now know that the file descriptor 8 is coming from WPE and is not internal to the EGL library. libwpe gets this file descriptor from the UI process, as one of the many creation parameters that are passed via IPC to the nascent process in order to initialize it. Turns out that this file descriptor in particular, the so-called host client file descriptor, is the one that the freedesktop backend of libWPE, from here onwards WPEBackend-fdo, creates when a new client is set to connect to its Wayland display. In a nutshell, in presence of a new client, a Wayland display is supposed to create a pair of connected sockets, create a new client on the Display-side, give it one of the file descriptors, and pass the other one to the client process. Because this will be useful later on, let's see how is that currently implemented in WPEBackend-fdo.

    int pair[2];
    if (socketpair(AF_UNIX, SOCK_STREAM | SOCK_CLOEXEC, 0, pair) < 0)
        return -1;

    int clientFd = dup(pair[1]);
    close(pair[1]);

    wl_client_create(m_display, pair[0]);

The file descriptor we are tracking down is the client file descriptor, clientFd. So we now know what's going on in this socket: Wayland-specific communication. Let's enable Wayland debugging next, by running all relevant process with WAYLAND_DEBUG=1. We'll get back to that code fragment later on.

A Heisenbug is a Heisenbug is a Heisenbug

Turns out that enabling Wayland debugging output for a few processes is enough to alter the state of the system in such a way that the bug does not happen at all when doing manual testing. Thankfully the CI's reproducibility is much higher, so after waiting overnight for the CI to continuously run until it hit the bug, we have logs. What do the logs say?

WPEWebProcess[41]: initializeEGLDisplay() starting.
  -> wl_display@1.get_registry(new id wl_registry@2)
  -> wl_display@1.sync(new id wl_callback@3)

So the EGL library is trying to fetch the Wayland registry and it's doing a wl_display_sync() call afterwards, which will block until the server responds. That's where the blocking poll() call comes from. So, it turns out, the problem is not necessarily on this end of the Wayland socket, but perhaps on the other side, that is, in the so-called UI process (the main browser process). Why is the Wayland display not replying?

The loop

Something that is worth mentioning before we move on is how the WPEBackend-fdo Wayland display integrates with the system. This display is a nested display, with each web view a client, while it is itself a client of the system's Wayland display. This can be a bit confusing if you're not very familiar with how Wayland works, but fortunately there is good documentation about Wayland elsewhere.

The way that the Wayland display in the UI process of a WPEWebKit browser is integrated with the rest of the program, when it uses WPEBackend-fdo, is through the GLib main event loop. Wayland itself has an event loop implementation for servers, but for a GLib-powered application it can be useful to use GLib's and integrate Wayland's event processing with the different stages of the GLib main loop. That is precisely how WPEBackend-fdo is handling its clients' events. As discussed earlier, when a new client is created a pair of connected sockets are created and one end is given to Wayland to control communication with the client. GSourceFunc functions are used to integrate Wayland with the application main loop. In these functions, we make sure that whenever there are pending messages to be sent to clients, those are sent, and whenever any of the client sockets has pending data to be read, Wayland reads from them, and to dispatch the events that might be necessary in response to the incoming data. And here is where things start getting really strange, because after doing a bit of fprintf()-powered debugging inside the Wayland-GSourceFuncs functions, it became clear that the Wayland events from the clients were never dispatched, because the dispatch() GSourceFunc was not being called, as if there was nothing coming from any Wayland client. But how is that possible, if we already know that the web process client is actually trying to get the Wayland registry?

To move forward, one needs to understand how the GLib main loop works, in particular, with Unix file descriptor sources. A very brief summary of this is that, during an iteration of the main loop, GLib will poll file descriptors to see if there are any interesting events to be reported back to their respective sources, in which case the sources will decide whether to trigger the dispatch() phase. A simple source might decide in its dispatch() method to directly read or write from/to the file descriptor; a Wayland display source (as in our case), will call wl_event_loop_dispatch() to do this for us. However, if the source doesn't find any interesting events, or if the source decides that it doesn't want to handle them, the dispatch() invocation will not happen. More on the GLib main event loop in its API documentation.

So it seems that for some reason the dispatch() method is not being called. Does that mean that there are no interesting events to read from? Let's find out.

System call tracing

Here we resort to another helpful tool, strace. With strace we can try to figure out what is happening when the main loop polls file descriptors. The strace output is huge (because it takes easily over a hundred attempts to reproduce this), but we know already some of the calls that involve file descriptors from the code we looked at above, when the client is created. So we can use those calls as a starting point in when searching through the several MBs of logs. Fast-forward to the relevant logs.

socketpair(AF_UNIX, SOCK_STREAM|SOCK_CLOEXEC, 0, [128, 130]) = 0
dup(130)               = 131
close(130)             = 0
fcntl64(128, F_DUPFD_CLOEXEC, 0) = 130
epoll_ctl(34, EPOLL_CTL_ADD, 130, {EPOLLIN, {u32=1639599928, u64=1639599928}}) = 0

What we see there is, first, WPEBackend-fdo creating a new socket pair (128, 130) and then, when file descriptor 130 is passed to wl_client_create() to create a new client, Wayland adds that file descriptor to its epoll() instance for monitoring clients, which is referred to by file descriptor 34. This way, whenever there are events in file descriptor 130, we will hear about them in file descriptor 34.

So what we would expect to see next is that, after the web process is spawned, when a Wayland client is created using the passed file descriptor and the EGL driver requests the Wayland registry from the display, there should be a POLLIN event coming in file descriptor 34 and, if the dispatch() call for the source was called, a epoll_wait() call on it, as that is what wl_event_loop_dispatch() would do when called from the source's dispatch() method. But what do we have instead?

poll([{fd=30, events=POLLIN}, {fd=34, events=POLLIN}, {fd=59, events=POLLIN}, {fd=110, events=POLLIN}, {fd=114, events=POLLIN}, {fd=132, events=POLLIN}], 6, 0) = 1 ([{fd=34, revents=POLLIN}])
recvmsg(30, {msg_namelen=0}, MSG_DONTWAIT|MSG_CMSG_CLOEXEC) = -1 EAGAIN (Resource temporarily unavailable)

strace can be a bit cryptic, so let's explain those two function calls. The first one is a poll in a series of file descriptors (including 30 and 34) for POLLIN events. The return value of that call tells us that there is a POLLIN event in file descriptor 34 (the Wayland display epoll() instance for clients). But unintuitively, the call right after is trying to read a message from socket 30 instead, which we know doesn't have any pending data at the moment, and consequently returns an error value with an errno of EAGAIN (Resource temporarily unavailable).

Why is the GLib main loop triggering a read from 30 instead of 34? And who is 30?

We can answer the latter question first. Breaking on a running UI process instance at the right time shows who is reading from the file descriptor 30:

#1  0x70ae1394 in wl_os_recvmsg_cloexec (sockfd=30, msg=msg@entry=0x700fea54, flags=flags@entry=64)
#2  0x70adf644 in wl_connection_read (connection=0x6f70b7e8)
#3  0x70ade70c in read_events (display=0x6f709c90)
#4  wl_display_read_events (display=0x6f709c90)
#5  0x70277d98 in pwl_source_check (source=0x6f71cb80)
#6  0x743f2140 in g_main_context_check (context=context@entry=0x2111978, max_priority=, fds=fds@entry=0x6165f718, n_fds=n_fds@entry=4)
#7  0x743f277c in g_main_context_iterate (context=0x2111978, block=block@entry=1, dispatch=dispatch@entry=1, self=)
#8  0x743f2ba8 in g_main_loop_run (loop=0x20ece40)
#9  0x00537b38 in ?? ()

So it's also Wayland, but on a different level. This is the Wayland client source (remember that the browser is also a Wayland client?), which is installed by cog (a thin browser layer on top of WPE WebKit that makes writing browsers easier to do) to process, among others, input events coming from the parent Wayland display. Looking at the cog code, we can see that the wl_display_read_events() call happens only if GLib reports that there is a G_IO_IN (POLLIN) event in its file descriptor, but we already know that this is not the case, as per the strace output. So at this point we know that there are two things here that are not right:

  1. A FD source with a G_IO_IN condition is not being dispatched.
  2. A FD source without a G_IO_IN condition is being dispatched.

Someone here is not telling the truth, and as a result the main loop is dispatching the wrong sources.

The loop (part II)

It is at this point that it would be a good idea to look at what exactly the GLib main loop is doing internally in each of its stages and how it tracks the sources and file descriptors that are polled and that need to be processed. Fortunately, debugging symbols for GLib are very small, so debugging this step by step inside the device is rather easy.

Let's look at how the main loop decides which sources to dispatch, since for some reason it's dispatching the wrong ones. Dispatching happens in the g_main_dispatch() method. This method goes over a list of pending source dispatches and after a few checks and setting the stage, the dispatch method for the source gets called. How is a source set as having a pending dispatch? This happens in g_main_context_check(), where the main loop checks the results of the polling done in this iteration and runs the check() method for sources that are not ready yet so that they can decide whether they are ready to be dispatched or not. Breaking into the Wayland display source, I know that the check() method is called. How does this method decide to be dispatched or not?

    [](GSource* base) -> gboolean
    {
        auto& source = *reinterpret_cast(base);
        return !!source.pfd.revents;
    },

In this lambda function we're returning TRUE or FALSE, depending on whether the revents field in the GPollFD structure have been filled during the polling stage of this iteration of the loop. A return value of TRUE indicates the main loop that we want our source to be dispatched. From the strace output, we know that there is a POLLIN (or G_IO_IN) condition, but we also know that the main loop is not dispatching it. So let's look at what's in this GPollFD structure.

For this, let's go back to g_main_context_check() and inspect the array of GPollFD structures that it received when called. What do we find?

(gdb) print *fds
$35 = {fd = 30, events = 1, revents = 0}
(gdb) print *(fds+1)
$36 = {fd = 34, events = 1, revents = 1}

That's the result of the poll() call! So far so good. Now the method is supposed to update the polling records it keeps and it uses when calling each of the sources check() functions. What do these records hold?

(gdb) print *pollrec->fd
$45 = {fd = 19, events = 1, revents = 0}
(gdb) print *(pollrec->next->fd)
$47 = {fd = 30, events = 25, revents = 1}
(gdb) print *(pollrec->next->next->fd)
$49 = {fd = 34, events = 25, revents = 0}

We're not interested in the first record quite yet, but clearly there's something odd here. The polling records are showing a different value in the revent fields for both 30 and 34. Are these records updated correctly? Let's look at the algorithm that is doing this update, because it will be relevant later on.

  pollrec = context->poll_records;
  i = 0;
  while (pollrec && i < n_fds)
    {
      while (pollrec && pollrec->fd->fd == fds[i].fd)
        {
          if (pollrec->priority <= max_priority)
            {
              pollrec->fd->revents =
                fds[i].revents & (pollrec->fd->events | G_IO_ERR | G_IO_HUP | G_IO_NVAL);
            }
          pollrec = pollrec->next;
        }

      i++;
    }

In simple words, what this algorithm is doing is to traverse simultaneously the polling records and the GPollFD array, updating the polling records revents with the results of polling. From reading how the pollrec linked list is built internally, it's possible to see that it's purposely sorted by increasing file descriptor identifier value. So the first item in the list will have the record for the lowest file descriptor identifier, and so on. The GPollFD array is also built in this way, allowing for a nice optimization: if more than one polling record – that is, more than one polling source – needs to poll the same file descriptor, this can be done at once. This is why this otherwise O(n^2) nested loop can actually be reduced to linear time.

One thing stands out here though: the linked list is only advanced when we find a match. Does this mean that we always have a match between polling records and the file descriptors that have just been polled? To answer that question we need to check how is the array of GPollFD structures filled. This is done in g_main_context_query(), as we hinted before. I'll spare you the details, and just focus on what seems relevant here: when is a poll record not used to fill a GPollFD?

  n_poll = 0;
  lastpollrec = NULL;
  for (pollrec = context->poll_records; pollrec; pollrec = pollrec->next)
    {
      if (pollrec->priority > max_priority)
        continue;
  ...

Interesting! If a polling record belongs to a source whose priority is lower than the maximum priority that the current iteration is going to process, the polling record is skipped. Why is this?

In simple terms, this happens because each iteration of the main loop finds out the highest priority between the sources that are ready in the prepare() stage, before polling, and then only those file descriptor sources with at least such a a priority are polled. The idea behind this is to make sure that high-priority sources are processed first, and that no file descriptor sources with lower priority are polled in vain, as they shouldn't be dispatched in the current iteration.

GDB tells me that the maximum priority in this iteration is -60. From an earlier GDB output, we also know that there's a source for a file descriptor 19 with a priority 0.

(gdb) print *pollrec
$44 = {fd = 0x7369c8, prev = 0x0, next = 0x6f701560, priority = 0}
(gdb) print *pollrec->fd
$45 = {fd = 19, events = 1, revents = 0}

Since 19 is lower than 30 and 34, we know that this record is before theirs in the linked list (and so it happens, it's the first one in the list too). But we know that, because its priority is 0, it is too low to be added to the file descriptor array to be polled. Let's look at the loop again.

  pollrec = context->poll_records;
  i = 0;
  while (pollrec && i < n_fds)
    {
      while (pollrec && pollrec->fd->fd == fds[i].fd)
        {
          if (pollrec->priority <= max_priority)
            {
              pollrec->fd->revents =
                fds[i].revents & (pollrec->fd->events | G_IO_ERR | G_IO_HUP | G_IO_NVAL);
            }
          pollrec = pollrec->next;
        }

      i++;
    }

The first polling record was skipped during the update of the GPollFD array, so the condition pollrec && pollrec->fd->fd == fds[i].fd is never going to be satisfied, because 19 is not in the array. The innermost while() is not entered, and as such the pollrec list pointer never moves forward to the next record. So no polling record is updated here, even if we have updated revent information from the polling results.

What happens next should be easy to see. The check() method for all polled sources are called with outdated revents. In the case of the source for file descriptor 30, we wrongly tell it there's a G_IO_IN condition, so it asks the main loop to call dispatch it triggering a a wl_connection_read() call in a socket with no incoming data. For the source with file descriptor 34, we tell it that there's no incoming data and its dispatch() method is not invoked, even when on the other side of the socket we have a client waiting for data to come and blocking in the meantime. This explains what we see in the strace output above. If the source with file descriptor 19 continues to be ready and with its priority unchanged, then this situation repeats in every further iteration of the main loop, leading to a hang in the web process that is forever waiting that the UI process reads its socket pipe.

The bug – explained

I have been using GLib for a very long time, and I have only fixed a couple of minor bugs in it over the years. Very few actually, which is why it was very difficult for me to come to accept that I had found a bug in one of the most reliable and complex parts of the library. Impostor syndrome is a thing and it really gets in the way.

But in a nutshell, the bug in the GLib main loop is that the very clever linear update of registers is missing something very important: it should skip to the first polling record matching before attempting to update its revents. Without this, in the presence of a file descriptor source with the lowest file descriptor identifier and also a lower priority than the cutting priority in the current main loop iteration, revents in the polling registers are not updated and therefore the wrong sources can be dispatched. The simplest patch to avoid this, would look as follows.

   i = 0;
   while (pollrec && i < n_fds)
     {
+      while (pollrec && pollrec->fd->fd != fds[i].fd)
+        pollrec = pollrec->next;
+
       while (pollrec && pollrec->fd->fd == fds[i].fd)
         {
           if (pollrec->priority <= max_priority)

Once we find the first matching record, let's update all consecutive records that also match and need an update, then let's skip to the next record, rinse and repeat. With this two-line patch, the web process was finally unlocked, the EGL display initialized properly, the web extension and the web page were loaded, CI tests starting passing again, and this exhausted developer could finally put his mind to rest.

A complete patch, including improvements to the code comments around this fascinating part of GLib and also a minimal test case reproducing the bug have already been reviewed by the GLib maintainers and merged to both stable and development branches. I expect that at least some GLib sources will start being called in a different (but correct) order from now on, so keep an eye on your GLib sources. :-)

Standing on the shoulders of giants

At this point I should acknowledge that without the support from my colleagues in the WebKit team in Igalia, getting to the bottom of this problem would have probably been much harder and perhaps my sanity would have been at stake. I want to thank Adrián and &Zcaronan for their input on Wayland, debugging techniques, and for allowing me to bounce back and forth ideas and findings as I went deeper into this rabbit hole, helping me to step out of dead-ends, reminding me to use tools out of my everyday box, and ultimately, to be brave enough to doubt GLib's correctness, something that much more often than not I take for granted.

Thanks also to Philip and Sebastian for their feedback and prompt code review!

October 29, 2020 01:10 PM



October 23, 2020

WebKitGTK 2.30.2 released!

by The WebKitGTK Project

This is a bug fix release in the stable 2.30 series.

What’s new in the WebKitGTK 2.30.2 release?

  • Fix scrolling when smooth scrolling is enabled in async scrolling mode.
  • Fix WebSocket requests with same-site cookies.
  • Fix TLS certificate information for service workers.
  • Handle chassis type when its value is quoted.
  • Fix the build with ENABLE_MEDIA_STREAM enabled and ENABLE_WEB_RTC_DISABLED.
  • Fix the build with ENABLE_GAMEPAD enabled.
  • Fix several crashes and rendering issues.

Thanks to all the contributors who made possible this release.

October 23, 2020 12:00 AM



October 01, 2020

Closing the gap (in flexbox 😇)

by Sergio Villar

Flexbox had a lot of early problems, but by mid-May 2020 where our story begins, both Firefox and Chromium had done a lot of work on improving things with this feature. WebKit, however, hadn’t caught up. Prioritizing the incredible amounts of work a web engine requires is difficult. The WebKit implementation was still passable for very many (most) cases of the core features, and it didn’t have problems that caused crashes or something that urgently demanded attention, so engineers dedicated their limited time toward other things. The net result, however, was that as this choice repeated many times, the comparative state of WebKit’s flexbox implementation had fallen behind pretty significantly.
Web Platform Tests (WPT) is a huge ongoing effort from many people to come up with a very extensive list of tests that could help both spec editors and implementors to make sure we have great compatibility. In the case of flexbox, for example, there are currently 773 tests (2926 subtests) and WebKit was failing a good amount of them. This matters a lot because there are things that flexbox is ideal for, and it is exceptionally widely used. In mid-May, Igalia was contracted to improve things here, and in this post, I’ll explain and illustrate how we did that.

The Challenge

The main issues were (in no particular order):
  • min-width:auto and min-height:auto handling
  • Nested flexboxes in column flows
  • Flexboxes inside tables and viceversa
  • Percentages in heights with indefinite sizes
  • WebKit CI not runnning many WPT flexbox tests
  • and of course… lack of gap support in Flexbox
Modifying Flexbox layout code is a challenge by itself. Tiny modifications in the source code could cause huge differences in the final layout. You might even have a patch that passes all the tests and regresses multiple popular web sites.
Good news is that we were able to tackle most of those issues. Let’s review what changes you could eventually expect from future releases of Safari (note that Apple doesn’t disclose information about future products and/or releases) and the other WebKit based browsers (like GNOME Web).

Flexbox gaps 🥳🎉

Probably one of the most awaited features in WebKit by web developers. It’s finally here after Firefox and Chrome landed it not so long ago. The implementation was initially inspired by the one in Chrome but then it diverged a bit in the final version of the patch. The important thing is that the behaviour should be the same, at least all the tests in WPT related to gaps are passing now in WebKit trunk.
<div style="display: flex; flex-wrap: wrap; gap: 1ch">
  <div style="background: magenta; color: white">Lorem</div>
  <div style="background: green; color: white">ipsum</div>
  <div style="background: orange; color: white">dolor</div>
  <div style="background: blue; color: white">sit</div>
  <div style="background: brown; color: white">amet</div>
</div>

Tables as flex items

Tables should obey the flex container sizing whenever they are flex items. As it can be seen in the examples bellow, the tables’ layout code was kicking in and ignoring the constraints set by the flex container. Tables should do what the flex algorithm mandates and thus they should allow being stretched/squeezed as required.
<div style="display:flex; width:100px; background:red;">
  <div style="display:table; width:10px; max-width:10px; height:100px; background:green;">
    <div style="width:100px; height:10px; background:green;"></div>
  </div>
</div>

Tables with items exceeding the 100% of available size

This is the case of tables placed inside flex items. The automatic layout table algorithm was generating tables with unlimited widths when the sum of the sizes of their columns (expressed in percentages) was exceeding the 100%. It was impossible to fulfill at the same time the constraints set by tables and flexbox algorithms.
<div style="display:flex; width:100px; height:100px; align-items:flex-start; background:green;">
  <div style="flex-grow:1; flex-shrink:0;">
    <table style="height:50px; background:green;" cellpadding="0" cellspacing="0">
      <tr>
        <td style="width:100%; background:green;"> </td>
        <td style="background:green;"> </td>
      </tr>
    </table>
  </div>
</div>

Note how the table was growing indefinitely (I cropped the “Before” picture to fit in the post) to the right before the fix.

Alignment in single-line flexboxes

Interesting case. The code was considering that single-line flexboxes were those where all the flex items were placed in a single line after computing the required space for them. Though sensible, that’s not what a single line flexbox is, it’s a flex container with flex-wrap:nowrap. This means that a flex container with flex-wrap:wrap whose children do not need more than 1 flex line to be placed is not a single-line flex container from the specs POV (corolary: implementing specs is hard).
<div style="display: flex; flex-wrap: wrap; align-content: flex-end; width: 425px; height: 70px; border: 2px solid black">
  <div style="height: 20px">This text should be at the bottom of its container</div>
</div>

Percentages in flex items with indefinite sizes

One of the trickiest ones. Although it didn’t involve a lot of code it caused two serious regressions in Youtube’s upload form and when viewing Twitter videos in fullscreen which required some previous fixes and delayed a bit the landing of this patch. Note that this behaviour was really conflictive from the pure specification POV as there were many changes over the time. Defining a good behaviour is really complicated. Without entering in too much details, flexbox has a couple of cases were sizes are considered as definite when they are theoretically indefinite. In this case we consider that if the flex container main size is definite then the post-flexing size of flex items is also treated as definite.
<div style="display: flex; flex-direction: column; height: 150px; width: 150px; border: 2px solid black;">
  <div>
    <div style="height: 50%; overflow: hidden;">
      <div style="width: 50px; height: 50px; background: green;"></div>
    </div>
  <div>
  <div style="flex: none; width: 50px; height: 50px; background: green;"></div>
</div>

Hit testing with overlapping flex items

There were some issues with pointer events passing through overlapping flex items (due to negative margins for example). This was fixed by letting the hit testing code proceed in reverse (the opposite to painting) order-modified document order instead of using the raw order from the DOM.
<div style="display:flex; border: 1px solid black; width: 300px;">
  <a style="width: 200px;" href="#">Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua</a>
  <div style="margin-left: -200px; width: 130px; height: 50px; background: orange;"></div>
</div>

In the “Before” case hit testing was bypassing the orange block and thus, the cursor was showing a hand because it detected that it was hovering a link. After the fix, the cursor is properly rendered as an arrow because the orange block covers the underneath link.

Computing percentages with scrollbars

In this case the issue was that, in order to compute percentages in heights, we were incorrectly using the size of the scrollbars too.
<div style="display: inline-flex; height: 10em;">
  <div style="overflow-x: scroll;">
    <div style="width: 200px; height: 100%; background: green"></div>
  </div>
</div>

Note that in the “After” picture the horizontal scrollbar background is visible while in the “Before” the wrong height computation made the flex item overlap the scrollbar.

Image items with specific sizes

The flex layout algorithm needs the intrinsic sizes of the flex items to compute their sizes and the size of the flex container. Changes to those intrinsic sizes should trigger new layouts, and the code was not doing that.
<!-- Just to showcase how the img bellow is not properly sized -->
<div style="position: absolute; background-color: red; width: 50px; height: 50px; z-index: -1;"></div>
 
<div style="display: flex; flex-direction: column; width: 100px; height: 5px;">
  <img style="width: 100px; height: 100px;" src="https://wpt.live/css/css-flexbox/support/100x100-green.png">
</div>


Nested flexboxes with ‘min-height: auto’

Another tricky one and another one related to the handling of nested column flexboxes. As in the previous issue with nested column flexboxes the problem was that we were not supporting this case. For those wanting to have a deeper understanding of the issue this bug was about implementing section 4.5 of the specs. This was one of the more complicated ones to fix, Edward Lorenz would love that part of the layout code, the slightest change in one of those source code lines could trigger huge changes in the final rendering.
<div style='display:flex; flex-direction: column; overflow-y: scroll; width: 250px; height: 250px; border: 1px solid black'>
  <div style='display:flex;'>
    <div style="width: 100px; background: blue"></div>
    <div style='width: 120px; background: orange'></div>
    <div style='width: 10px; background: yellow; height: 300px'></div>
  </div>
</div>

As it can be seen, in the “Before” picture the blue and orange blocks are sized differently to the yellow one. That’s fixed in the “After” picture.

Percentages in quirks mode

Another one affecting how percentages are computed in heights, but this one specific to quirks mode. We’re matching now Firefox, Chrome and pre-Chromium Edge, i.e., flexbox should not care much about quirks mode since it was invented many years after quirky browsers dominated the earth.
<!DOCTYPE html PUBLIC>
<div style="width: 100px; height: 50px;">
  <div style="display: flex; flex-direction: column; outline: 2px solid blue;">
    <div style="flex: 0 0 50%"></div>
  </div>
</div>

Percentages in ‘flex-basis’

Percentages were working generally fine inside flex-basis, however there was one particular problematic case. It arose whenever that percentage was refererring to, oh surprise, and indefinite height. And again, we’re talking about nested flexboxes with column flows. Indeed, definite/indefinite sizes is one of the toughest things to get right from the layout POV. In this particular case, the fix was to ignore the percentages and and treat them as height: auto.
<div style="display: flex; flex-direction: column; width: 200px;">
  <div style="flex-basis: 0%; height: 100px; background: red;">
    <div style="background: lime">Here's some text.</div>
  </div>
</div>

Flex containers inside STF tables

Fixing a couple of test cases submitted by an anonymous Opera employee 8! years ago. This is another case of competing layout contexts trying to do things their own way.
<div style="display: table; background:red">
   <div style="display: flex; width: 0px">
      <p style="margin: 1em 1em;width: 50px">Text</p>
      <p style="margin: 1em 1em;width: 50px">Text</p>
      <p style="margin: 1em 1em;width: 50px">Text</p>
   </div>
</div>

After the fix the table is properly sized to 0px width and thus no red is seen.

Conclusions

These examples are just some interesting ones I’ve chosen to highlight. In the end, almost 50 new flexbox tests are passing in WebKit that weren’t back in May!. I wouldn’t like to forget the great job done by my colleague Carlos Lopez who imported tons of WPT flexbox tests into the WebKit source tree. He also performed awesome triage work which made my life a lot easier.
Investing in interoperability is a huge deal for the web. It’s good for everyone, from spec authors to final users, including browser vendors, downstream ports or web authors. So if you care about the web, or your business orbits around web technologies, you should definitely promote and invest on interoperability.

Implementing standards or fixing bugs in web engines is the kind of work we happily do at Igalia on a daily basis. We are the second largest contributor to both WebKit and Chrome/Blink, so if you have an annoying bug on a particular web engine (Gecko and Servo as well) that you want to be fixed, don’t hesitate and contact us, we’d be glad to help. Also, should you want to be part of a workers-owned cooperative with an asambleary decision-making mechanism and a strong focus on free software technologies join us!.

Acknowledgements

Many thanks to WebKit reviewers from Apple and Igalia like Darin Adler, Manuel Rego, Javier Fernández or Daniel Bates who made the process really easy for me, always providing very nice feedback for the patches I submitted.
I’m also really thankful to Googlers like Christian Biesinger, David Grogan and Stephen McGruer who worked on the very same things in Blink and/or provided very nice guidance and support when porting patches.

by svillar at October 01, 2020 11:34 AM



September 21, 2020

WebKitGTK 2.30.1 released!

by The WebKitGTK Project

This is the first bug fix release in the stable 2.30 series.

What’s new in the WebKitGTK 2.30.1 release?

  • Bring back the environment variable to force single process mode when PSON is disabled.
  • Fix downloads started by an ephemeral web context.
  • Translation updates: Brazilian Portuguese.

Thanks to all the contributors who made possible this release.

September 21, 2020 12:00 AM



September 16, 2020

Epiphany 3.38 and WebKitGTK 2.30

by Michael Catanzaro

It’s that time of year again: a new GNOME release, and with it, a new Epiphany. The pace of Epiphany development has increased significantly over the last few years thanks to an increase in the number of active contributors. Most notably, Jan-Michael Brummer has solved dozens of bugs and landed many new enhancements, Alexander Mikhaylenko has polished numerous rough edges throughout the browser, and Andrei Lisita has landed several significant improvements to various Epiphany dialogs. That doesn’t count the work that Igalia is doing to maintain WebKitGTK, the WPE graphics stack, and libsoup, all of which is essential to delivering quality Epiphany releases, nor the work of the GNOME localization teams to translate it to your native language. Even if Epiphany itself is only the topmost layer of this technology stack, having more developers working on Epiphany itself allows us to deliver increased polish throughout the user interface layer, and I’m pretty happy with the result. Let’s take a look at what’s new.

Intelligent Tracking Prevention

Intelligent Tracking Prevention (ITP) is the headline feature of this release. Safari has had ITP for several years now, so if you’re familiar with how ITP works to prevent cross-site tracking on macOS or iOS, then you already know what to expect here.  If you’re more familiar with Firefox’s Enhanced Tracking Protection, or Chrome’s nothing (crickets: chirp, chirp!), then WebKit’s ITP is a little different from what you’re used to. ITP relies on heuristics that apply the same to all domains, so there are no blocklists of naughty domains that should be targeted for content blocking like you see in Firefox. Instead, a set of innovative restrictions is applied globally to all web content, and a separate set of stricter restrictions is applied to domains classified as “prevalent” based on your browsing history. Domains are classified as prevalent if ITP decides the domain is capable of tracking your browsing across the web, or non-prevalent otherwise. (The public-friendly terminology for this is “Classification as Having Cross-Site Tracking Capabilities,” but that is a mouthful, so I’ll stick with “prevalent.” It makes sense: domains that are common across many websites can track you across many websites, and domains that are not common cannot.)

ITP is enabled by default in Epiphany 3.38, as it has been for several years now in Safari, because otherwise only a small minority of users would turn it on. ITP protections are designed to be effective without breaking too many websites, so it’s fairly safe to enable by default. (You may encounter a few broken websites that have not been updated to use the Storage Access API to store third-party cookies. If so, you can choose to turn off ITP in the preferences dialog.)

For a detailed discussion covering ITP’s tracking mitigations, see Tracking Prevention in WebKit. I’m not an expert myself, but the short version is this: full third-party cookie blocking across all websites (to store a third-party cookie, websites must use the Storage Access API to prompt the user for permission); cookie-blocking latch mode (“once a request is blocked from using cookies, all redirects of that request are also blocked from using cookies”); downgraded third-party referrers (“all third-party referrers are downgraded to their origins by default”) to avoid exposing the path component of the URL in the referrer; blocked third-party HSTS (“HSTS […] can only be set by the first-party website […]”) to stop abuse by tracker scripts; detection of cross-site tracking via link decoration and 24-hour expiration time for all cookies created by JavaScript on the landing page when detected; a 7-day expiration time for all other cookies created by JavaScript (yes, this applies to first-party cookies); and a 7-day extendable lifetime for all other script-writable storage, extended whenever the user interacts with the website (necessary because tracking companies began using first-party scripts to evade the above restrictions). Additionally, for prevalent domains only, domains engaging in bounce tracking may have cookies forced to SameSite=strict, and Verified Partitioned Cache is enabled (cached resources are re-downloaded after seven days and deleted if they fail certain privacy tests). Whew!

WebKit has many additional privacy protections not tied to the ITP setting and therefore not discussed here — did you know that cached resources are partioned based on the first-party domain? — and there’s more that’s not very well documented which I don’t understand and haven’t mentioned (tracker collusion!), but that should give you the general idea of how sophisticated this is relative to, say, Chrome (chirp!). Thanks to John Wilander from Apple for his work developing and maintaining ITP, and to Carlos Garcia for getting it working on Linux. If you’re interested in the full history of how ITP has evolved over the years to respond to the changing threat landscape (e.g. tracking prevention tracking), see John’s WebKit blog posts. You might also be interested in WebKit’s Tracking Prevention Policy, which I believe is the strictest anti-tracking stance of any major web engine. TL;DR: “we treat circumvention of shipping anti-tracking measures with the same seriousness as exploitation of security vulnerabilities. If a party attempts to circumvent our tracking prevention methods, we may add additional restrictions without prior notice.” No exceptions.

Updated Website Data Preferences

As part of the work on ITP, you’ll notice that Epiphany’s cookie storage preferences have changed a bit. Since ITP enforces full third-party cookie blocking, it no longer makes sense to have a separate cookie storage preference for that, so I replaced the old tri-state cookie storage setting (always accept cookies, block third-party cookies, block all cookies) with two switches: one to toggle ITP, and one to toggle all website data storage.

Previously, it was only possible to block cookies, but this new setting will additionally block localStorage and IndexedDB, web features that allow websites to store arbitrary data in your browser, similar to cookies. It doesn’t really make much sense to block cookies but allow other types of data storage, so the new preferences should better enforce the user’s intent behind disabling cookies. (This preference does not yet block media keys, service workers, or legacy offline web application cache, but it probably should.) I don’t really recommend disabling website data storage, since it will cause compatibility issues on many websites, but this option is there for those who want it. Disabling ITP is also not something I want to recommend, but it might be necessary to access certain broken websites that have not yet been updated to use the Storage Access API.

Accordingly, Andrei has removed the old cookies dialog and moved cookie management into the Clear Personal Data dialog, which is a better place because anyone clearing cookies for a particular website is likely to also want to clear other personal data. (If you want to delete a website’s cookies, then you probably don’t want to leave its SQL databases intact, right?) He had to remove the ability to clear data from a particular point in time, because WebKit doesn’t support this operation for cookies, but that function is probably  rarely-used and I think the benefit of the change should outweigh the cost. (We could bring it back in the future if somebody wants to try implementing that feature in WebKit, but I suspect not many users will notice.) Treating cookies as separate and different from other forms of website data storage no longer makes sense in 2020, and it’s good to have finally moved on from that antiquated practice.

New HTML Theme

Carlos Garcia has added a new Adwaita-based HTML theme to WebKitGTK 2.30, and removed support for rendering HTML elements using the GTK theme (except for scrollbars). Trying to use the GTK theme to render web content was fragile and caused many web compatibility problems that nobody ever managed to solve. The GTK developers were never very fond of us doing this in the first place, and the foreign drawing API required to do so has been removed from GTK 4, so this was also good preparation for getting WebKitGTK ready for GTK 4. Carlos’s new theme is similar to Adwaita, but gradients have been toned down or removed in order to give a flatter, neutral look that should blend in nicely with all pages while still feeling modern.

This should be a fairly minor style change for Adwaita users, but a very large change for anyone using custom themes. I don’t expect everyone will be happy, but please trust that this will at least result in better web compatibility and fewer tricky theme-related bug reports.

Screenshot demonstrating new HTML theme vs. GTK theme
Left: Adwaita GTK theme controls rendered by WebKitGTK 2.28. Right: hardcoded Adwaita-based HTML theme with toned down gradients.

Although scrollbars will still use the GTK theme as of WebKitGTK 2.30, that will no longer be possible to do in GTK 4, so themed scrollbars are almost certain to be removed in the future. That will be a noticeable disappointment in every app that uses WebKitGTK, but I don’t see any likely solution to this.

Media Permissions

Jan-Michael added new API in WebKitGTK 2.30 to allow muting individual browser tabs, and hooked it up in Epiphany. This is good when you want to silence just one annoying tab without silencing everything.

Meanwhile, Charlie Turner added WebKitGTK API for managing autoplay policies. Videos with sound are now blocked from autoplaying by default, while videos with no sound are still allowed. Charlie hooked this up to Epiphany’s existing permission manager popover, so you can change the behavior for websites you care about without affecting other websites.

Screenshot displaying new media autoplay permission settings
Configure your preferred media autoplay policy for a website near you today!

Improved Dialogs

In addition to his work on the Clear Data dialog, Andrei has also implemented many improvements and squashed bugs throughout each view of the preferences dialog, the passwords dialog, and the history dialog, and refactored the code to be much more maintainable. Head over to his blog to learn more about his accomplishments. (Thanks to Google for sponsoring Andrei’s work via Google Summer of Code, and to Alexander for help mentoring.)

Additionally, Adrien Plazas has ported the preferences dialog to use HdyPreferencesWindow, bringing a pretty major design change to the view switcher:

Screenshot showing changes to the preferences dialog
Left: Epiphany 3.36 preferences dialog. Right: Epiphany 3.38. Note the download settings are present in the left screenshot but missing from the right screenshot because the right window is using flatpak, and the download settings are unavailable in flatpak.

User Scripts

User scripts (like Greasemonkey) allow you to run custom JavaScript on websites. WebKit has long offered user script functionality alongside user CSS, but previous versions of Epiphany only exposed user CSS. Jan-Michael has added the ability to configure a user script as well. To enable, visit the Appearance tab in the preferences dialog (a somewhat odd place, but it really needs to be located next to user CSS due to the tight relationship there). Besides allowing you to do, well, basically anything, this also significantly enhances the usability of user CSS, since now you can apply certain styles only to particular websites. The UI is a little primitive — your script (like your CSS) has to be one file that will be run on every website, so don’t try to design a complex codebase using your user script — but you can use conditional statements to limit execution to specific websites as you please, so it should work fairly well for anyone who has need of it. I fully expect 99.9% of users will never touch user scripts or user styles, but it’s nice for power users to have these features available if needed.

HTTP Authentication Password Storage

Jan-Michael and Carlos Garcia have worked to ensure HTTP authentication passwords are now stored in Epiphany’s password manager rather than by WebKit, so they can now be viewed and deleted from Epiphany, which required some new WebKitGTK API to do properly. Unfortunately, WebKitGTK saves network passwords using the default network secret schema, meaning its passwords (saved by older versions of Epiphany) are all leaked: we have no way to know which application owns those passwords, so we don’t have any way to know which passwords were stored by WebKit and which can be safely managed by Epiphany going forward. Accordingly, all previously-stored HTTP authentication passwords are no longer accessible; you’ll have to use seahorse to look them up manually if you need to recover them. HTTP authentication is not very commonly-used nowadays except for internal corporate domains, so hopefully this one-time migration snafu will not be a major inconvenience to most users.

New Tab Animation

Jan-Michael has added a new animation when you open a new tab. If the newly-created tab is not visible in the tab bar, then the right arrow will flash to indicate success, letting you know that you actually managed to open the page. Opening tabs out of view happens too often currently, but at least it’s a nice improvement over not knowing whether you actually managed to open the tab or not. This will be improved further next year, because Alexander is working on a completely new tab widget to replace GtkNotebook.

New View Source Theme

Jim Mason changed view source mode to use a highlight.js theme designed to mimic Firefox’s syntax highlighting, and added dark mode support.

Image showing dark mode support in view source mode
Embrace the dark.

And More…

  • WebKitGTK 2.30 now supports video formats in image elements, thanks to Philippe Normand. You’ll notice that short GIF-style videos will now work on several major websites where they previously didn’t.
  • I added a new WebKitGTK 2.30 API to expose the paste as plaintext editor command, which was previously internal but fully-functional. I’ve hooked it up in Epiphany’s context menu as “Paste Text Only.” This is nice when you want to discard markup when pasting into a rich text editor (such as the WordPress editor I’m using to write this post).
  • Jan-Michael has implemented support for reordering pinned tabs. You can now drag to reorder pinned tabs any way you please, subject to the constraint that all pinned tabs stay left of all unpinned tabs.
  • Jan-Michael added a new import/export menu, and the bookmarks import/export features have moved there. He also added a new feature to import passwords from Chrome. Meanwhile, ignapk added support for importing bookmarks from HTML (compatible with Firefox).
  • Jan-Michael added a new preference to web apps to allow running them in the background. When enabled, closing the window will only hide the the window: everything will continue running. This is useful for mail apps, music players, and similar applications.
  • Continuing Jan-Michael’s list of accomplishments, he removed Epiphany’s previous hidden setting to set a mobile user agent header after discovering that it did not work properly, and replaced it by adding support in WebKitGTK 2.30 for automatically setting a mobile user agent header depending on the chassis type detected by logind. This results in a major user experience improvement when using Epiphany as a mobile browser. Beware: this functionality currently does not work in flatpak because it requires the creation of a new desktop portal.
  • Stephan Verbücheln has landed multiple fixes to improve display of favicons on hidpi displays.
  • Zach Harbort fixed a rounding error that caused the zoom level to display oddly when changing zoom levels.
  • Vanadiae landed some improvements to the search engine configuration dialog (with more to come) and helped investigate a crash that occurs when using the “Set as Wallpaper” function under Flatpak. The crash is pretty tricky, so we wound up disabling that function under Flatpak for now. He also updated screenshots throughout the  user help.
  • Sabri Ünal continued his effort to document and standardize keyboard shortcuts throughout GNOME, adding a few missing shortcuts to the keyboard shortcuts dialog.

Epiphany 3.38 will be the final Epiphany 3 release, concluding a decade of releases that start with 3. We will match GNOME in following a new version scheme going forward, dropping the leading 3 and the confusing even/odd versioning. Onward to Epiphany 40!

by Michael Catanzaro at September 16, 2020 05:00 PM



September 11, 2020

WebKitGTK 2.30.0 released!

by The WebKitGTK Project

This is the first stable release in the 2.30 series.

Highlights of the WebKitGTK 2.30.0 release

  • Add Intelligent Tracking Prevention (ITP) support.
  • Add support for backdrop filters.
  • Stop using GTK theming to render form controls and add API to disable using GTK to render scrollbars too.
  • Add support for video formats in image elements.
  • Add API to handle video autoplay policy that now defaults to disallow autoplay videos with audio.
  • Add API to mute a web view.
  • Add paste as plain text option to the context menu for rich editable content.

For more details about all the changes included in WebKitGTK 2.30 see the NEWS file that is included in the tarball.

Thanks to all the contributors who made possible this release.

September 11, 2020 12:00 AM



September 07, 2020

Review of Igalia Multimedia activities (2020/H1)

by Víctor Jáquez

This blog post is a review of the various activities the Igalia Multimedia team was involved in during the first half of 2020.

Our previous reports are:

Just before a new virus turned into pandemics we could enjoy our traditional FOSDEM. There, our colleague Phil gave a talk about many of the topics covered in this report.

GstWPE

GstWPE’s wpesrc element, produces a video texture representing a web page rendered off-screen by WPE.

We have worked on a new iteration of the GstWPE demo, focusing on one-to-many, web-augmented overlays, broadcasting with WebRTC and Janus.

Also, since the merge of gstwpe plugin in gst-plugins-bad (staging area for new elements) new users have come along spotting rough areas and improving the element along the way.

Video Editing

GStreamer Editing Services (GES) is a library that simplifies the creation of multimedia editing applications. It is based on the GStreamer multimedia framework and is heavily used by Pitivi video editor.

Implemented frame accuracy in the GStreamer Editing Services (GES)

As required by the industry, it is now possible to reference all time in frame number, providing a precise mapping between frame number and play time. Many issues were fixed in GStreamer to reach the precision enough for make this work. Also intensive regression tests were added.

Implemented time effects support in GES

Important refactoring inside GStreamer Editing Services have happened to allow cleanly and safely change playback speed of individual clips.

Implemented reverse playback in GES

Several issues have been fixed inside GStreamer core elements and base classes in order to support reverse playback. This allows us to implement reliable and frame accurate reverse playback for individual clips.

Implemented ImageSequence support in GStreamer and GES

Since OpenTimelineIO implemented ImageSequence support, many users in the community had said it was really required. We reviewed and finished up imagesequencesrc element, which had been awaiting review for years.

This feature is now also supported in the OpentimelineIO GES adapater.

Optimized nested timelines preroll time by an order of magnitude

Caps negotiation, done while the pipeline transitions from pause state to playing state, testing the whole pipeline functionality, was the bottleneck for nested timelines, so pipelines were reworked to avoid useless negotiations. At the same time, other members of GStreamer community have improved caps negotiation performance in general.

Last but not least, our colleague Thibault gave a talk in The Pipeline Conference about The Motion Picture Industry and Open Source Software: GStreamer as an Alternative, explaining how and why GStreamer could be leveraged in the motion picture industry to allow faster innovation, and solve issues by reusing all the multi-platform infrastructure the community has to offer.

WebKit multimedia

There has been a lot of work on WebKit multimedia, particularly for WebKitGTK and WPE ports which use GStreamer framework as backend.

WebKit Flatpak SDK

But first of all we would like to draw readers attention to the new WebKit Flatpak SDK. It was not a contribution only from the multimedia team, but rather a joint effort among different teams in Igalia.

Before WebKit Flatpak SDK, JHBuild was used for setting up a WebKitGTK/WPE environment for testing and development. Its purpose to is to provide a common set of well defined dependencies instead of relying on the ones available in the different Linux distributions, which might bring different outputs. Nonetheless, Flatpak offers a much more coherent environment for testing and develop, isolated from the rest of the building host, approaching to reproducible outputs.

Another great advantage of WebKit Flatpak SDK, at least for the multimedia team, is the possibility of use gst-build to setup a custom GStreamer environment, with latest master, for example.

Now, for sake of brevity, let us sketch an non-complete list of activities and achievements related with WebKit multimedia.

General multimedia

Media Source Extensions (MSE)

Encrypted Media Extension (EME)

One of the major results of this first half, is the upstream of ThunderCDM, which is an implementation of a Content Decryption Module, providing Widevine decryption support. Recently, our colleague Xabier, published a blog post on this regard.

And it has enabled client-side video rendering support, which ensures video frames remain protected in GPU memory so they can’t be reached by third-party. This is a requirement for DRM/EME.

WebRTC

GStreamer

Though we normally contribute in GStreamer with the activities listed above, there are other tasks not related with WebKit. Among these we can enumerate the following:

GStreamer VAAPI

  • Reviewed a lot of patches.
  • Support for media-driver (iHD), the new VAAPI driver for Intel, mostly for Gen9 onwards. There are a lot of features with this driver.
  • A new vaapioverlay element.
  • Deep code cleanups. Among these we would like to mention:
    • Added quirk mechanism for different backends.
    • Change base classes to GstObject and GstMiniObject of most of classes and buffers types.
  • Enhanced caps negotiation given current driver’s constraints

Conclusions

The multimedia team in Igalia has keep working, along the first half of this strange year, in our three main areas: browsers (mainly on WebKitGTK and WPE), video editing and GStreamer framework.

We worked adding and enhancing WebKitGTK and WPE multimedia features in order to offer a solid platform for media providers.

We have enhanced the Video Editing support in GStreamer.

And, along these tasks, we have contribuited as much in GStreamer framework, particulary in hardware accelerated decoding and encoding and VA-API.

by vjaquez at September 07, 2020 03:12 PM



September 04, 2020

WebKitGTK 2.29.92 released!

by The WebKitGTK Project

This is a development release leading toward 2.30 series.

What’s new in the WebKitGTK 2.29.92 release?

  • Fix user agent header after a redirect when a new quirk is required.
  • Stop using firefox user agent quirk for google docs.
  • Fix rendering frames timeline panel in web inspector.
  • Fix per-thread cpu usage in web inspector.
  • Fix several crashes and rendering issues.
  • Translation updates: Polish

Thanks to all the contributors who made possible this release.

September 04, 2020 12:00 AM



September 02, 2020

Serious Encrypted Media Extensions on GStreamer based WebKit ports

by Xabier Rodríguez Calvar

Encrypted Media Extensions (a.k.a. EME) is the W3C standard for encrypted media in the web. This way, media providers such as Hulu, Netflix, HBO, Disney+, Prime Video, etc. can provide their contents with a reasonable amount of confidence that it will make it very complicated for people to “save” their assets without their permission. Why do I use the word “serious” in the title? In WebKit there is already support for Clear Key, which is the W3C EME reference implementation but EME supports more encryption systems, even privative ones (I have my opinion about this, you can ask me privately). No service provider (that I know) supports Clear Key, they usually rely on Widevine, PlayReady or some other.

Three years ago, my colleague Žan Doberšek finished the implementation of what was going to be the shell of WebKit’s modern EME implementation, following latest W3C proposal. We implemented that downstream (at Web Platform for Embedded) as well using Thunder, which includes as a plugin a fork of what was Open Content Decryption Module (a.k.a. OpenCDM). The OpenCDM API changed quite a lot during this journey. It works well and there are millions of set-top-boxes using it currently.

The delta between downstream and the upstream GStreamer based WebKit ports was quite big, testing was difficult and syncing was not always easy, so we decided reverse the situation.

Our first step was done by my colleague Charlie Turner, who made Clear Key work upstream again while adapted some changes the Apple folks had done meanwhile. It was amazing to see Clear Key tests passing again and his work with the CDMProxy related classes was awesome. After having ClearKey working, I had to adapt them a bit to accomodate Thunder. To explain a bit about the WebKit EME architecture, I must say that there are two layers. The first is the crossplatform one, which implements the W3C API (MediaKeys, MediaKeySession, CDM…). These classes rely on the platform ones (CDMPrivate, CDMInstance, CDMInstanceSession) to handle the platform management, message exchange, etc. which would be the second layer. Apple playback system is fully integrated with their DRM system so they don’t need anything else. We do because we need to integrate our own decryptors to defer to Thunder for decryption so in the GStreamer based ports we also need the CDMProxy related classes, which would be CDMProxy, CDMInstanceProxy, CDMInstanceSessionProxy… The last two extend CDMInstance and CDMInstanceSession respectively to be able to deal with the key management, that is abstracted to the KeyHandle and KeyStore.

Once the abstraction is there (let’s remember that the abstranction works both for Clear Key and Thunder), the Thunder implementation is quite simple, just gluing the CDMProxy, CDMInstanceProxy and CDMInstanceSessionProxy classes to the Thunder system and writing a GStreamer decryptor element for it. I might have made a mistake when selecting the files but considering Thunder classes + the GStreamer common decryptor code, cloc says it is just 1198 lines of platform code. I think it is pretty low for what it does. Apart from that, obviously, there are 5760 lines of crossplatform code.

To build and run all this you need to do several things:

  1. Build the dependencies with WEBKIT_JHBUILD=1 JHBUILD_ENABLE_THUNDER="yes" to enable the old fashioned JHBuild build and force it to build the Thunder dependencies. All dependendies are on JHBuild, even Widevine is referenced but to download it you need the proper credentials as it is closed source.
  2. Pass --thunder when calling build-webkit.sh.
  3. Run MiniBrowser with WEBKIT_GST_EME_RANK_PRIORITY="Thunder" and pass parameters --enable-mediasource=TRUE --enable-encrypted-media=TRUE --autoplay-policy=allow. The autoplay policy is usually optional but in this case it is necessary for the YouTube TV tests. We need to give the Thunder decryptor a higher priority because of WebM, that does not specify a key system and without it the Clear Key one can be selected and fail. MP4 does not create trouble because the protection system is specified and the caps negotiation does its magic.

As you could have guessed if you have a closer look at the GStreamer JHBuild moduleset, you’ll see that only Widevine is supported. To support more, you only have to make them build in the Thunder ecosystem and add them to CDMFactoryThunder::supportedKeySystems.

When I coded this, all YouTube TV tests for Widevine were green in the desktop. At the moment of writing this post they aren’t because of some problem with the Widevine installation that will be sorted quickly, I hope.

by calvaris at September 02, 2020 02:59 PM



August 14, 2020

WebKitGTK 2.29.91 released!

by The WebKitGTK Project

This is a development release leading toward 2.30 series.

What’s new in the WebKitGTK 2.29.91 release?

  • Fix a web process crash introduced in 2.29.90.

Thanks to all the contributors who made possible this release.

August 14, 2020 12:00 AM



WebKitGTK 2.29.90 released!

by The WebKitGTK Project

This is a development release leading toward 2.30 series.

What’s new in the WebKitGTK 2.29.90 release?

  • Fix font variation settings when font smoothing setting is also present.
  • Fix HTML drag and drop operations.
  • Fix argument order for clone syscall seccomp filter on s390x.
  • Fix a crash when selecting text.
  • Fix several crashes and rendering issues.

Thanks to all the contributors who made possible this release.

August 14, 2020 12:00 AM



August 13, 2020

Improving CSS Custom Properties performance

by Javier Fernández

Chrome 84 reached the stable channel a few weeks ago, and there are already several great posts describing the many important additions, interesting new features, security fixes and improvements in privacy policies (([1], [2], [3], [4]) it contains. However, there is a change that I worked on in this release which might have passed unnoticed by most, but I think is very valuable: A change regarding CSS Custom Properties (variables) performance.

The design of CSS, in general, takes great care in considering how features are designed with respect to making it possible for them to perform well. However, implementations may not perform as well as they could, and it takes a considerable amount of time to understand how authors use the features and which cases are more relevant for them.

CSS Custom Properties are an interesting example to look at here: They are a wonderful feature that provides a lot of advantages for web authors. For a whole lot of cases, all of the implementations of CSS Custom Properties perform well enough that most people won’t notice. However, we at Igalia have been analyzing several use cases and looking at some reports around their performance in different implementations.

Let’s consider a fairly straightforward example in which an author sets a single property in a toggleable class in the body, and then uses that property several times deeper in the tree to change the foreground color of some text.


Only about 20% of those actually use this property, 5 elements deep into the tree, and only to change the foreground color.

To evaluate Chromium’s performance in a case like this we can define a new perf tests, using the perf tools the Chromium project has available for browser engineers. In this case, we want a huge tree so that we can evaluate better the impact of the different optimizations.

    

These are the results obtained runing the test in Chrome 83:

avg median stdev min max
163.74 ms 163.79 ms 3.69 ms 158.59 ms 163.74 ms

I admit that it’s difficult to evaluate the results, especially considering the number of nodes of such a huge DOM tree. Lets compare the results of the same test on Firefox, using different number of nodes.

Nodes 50K 20K 10K 5K 1K 500
Chrome 83 163.74 ms 55.05 ms 25.12 ms 14.18 ms 2.74 ms 1.50 ms
FF 78 28.35 ms 12.05 ms 6.10 ms 3.50 ms 1.15 ms 0.55 ms
1/6 1/5 1/4 1/4 1/2 1/3

As I commented before, the data are more accurate when the DOM tree has a lot of nodes; in any case, the difference is quite clear and shows there is plenty room for improvement. WebKit based browsers have results more similar to Chromium as well.

Performance tests like the one above can be added to browsers for tracking improvements and regressions over time, so we’ve added (r763335) that to Chromium’s tree: We’d like to see it get faster over time, and definitely cannot afford regressions (see Chrome Performance Dashboard and the ChangeStyleCustomPropertyDeclaration test for details) .

So… What can we do?

In Chrome 83 and lower, whenever the custom property declaration changed, the new declaration would be inherited by the whole tree. This inheritance implied executing the whole CSS cascade and recalculating the styles of all the nodes in the entire tree, since with this approach, all nodes may be affected.

Chrome had already implemented an optimization on the CSS cascade implementation for regular CSS properties that don’t depend on any other to resolve their value. These subset of CSS properties are defined as Independent Properties in the Chromium codebase. The optimization mentioned before affects how the inheritance mechanism is implemented for these Independent properties. Whenever one of these properties changes, instead of recalculating the styles of the inherited properties, children can just copy the whole parent’s computed style. Blink’s style engine has a component known as Matched Properties Cache responsible of deciding when is possible to avoid the style resolution of an element and instead, performing an efficient copy of the matched computed style. I’ll get back to this concept in the last part of this post.

In the case of CSS Custom Properties, we could apply a similar approach as a good step. We can consider that the nodes with computed styles that don’t have references to custom properties declarations shouldn’t be affected by the new declaration, and we can implement the inheritance directly by copying the parent’s computed style. The patch with the optimization I’ve implemented in r765278 initially landed in Chrome 84.0.4137.0

Let’s look at the result of this one action in the Chrome Performance Dashboard:

That’s a really good improvement!

However, it’s also just a first step. It’s clear that Chrome still has a wide margin for improvement in this case, as well any WebKit based browser – Firefox is still, impressively, markedly faster as it’s been described in the bug report filed to track this issue. The following table shows the result of the different browsers together; even disabling the muti-thread capabilities of Firefox’s Stylo engine (STYLO_THREAD=1), FF is much faster than Chrome with the optimization applied.

Chrome 83 Chrome 84 FF 78 FF 78 th=1
avg
median
stdev
min
max
163.74 ms
163.79 ms
3.69 ms
158.59 ms
163.74 ms
117.37 ms
117.52 ms
1.98 ms
113.66 ms
120.87 ms
28.35 ms
28.50 ms
0.93 ms
26.00 ms
30.00 ms
38.25 ms
38.50 ms
1.86 ms
35.00 ms
41.00 ms

Before continue, I want get back to the Matched Properties Cache (MPC) concept, since it has an important role on these style optimizations. This cache is not a new concept in the Chrome’s engine; as a matter of fact, it’s also used in WebKit, since it was implemented long ago, before the fork that created the new blink engine. However, Google has been working a lot on this area in the last years and some of the most recent changes in the MPC have had an important impact on style resolution performance. As a result of this work, elements with independent and non-independent properties using CSS Variables might produce cache hits in the MPC. The results of the Performance Dashboard show a considerable improvement in the mentioned ChangeStyleCustomPropertyDeclaration test (avg: 108.06 ms)

Additionally, there are several other cases where the use of CSS Variables has a considerable impact on performance, compared with using regular CSS properties. Obviously, resolving CSS Variables has a cost, so it’s clear that we could apply additional optimizations that reduce the impact of the variable resolution, especially for handling specific style changes that might not affect to a substantial portion of the DOM tree. I’ve been experimenting with the MPC to explore the idea an independent CSS Custom Properties cache; nodes with variables referencing the same custom property will produce cache hits in the MPC, even though other properties don’t match. The preliminary approach I’ve been implementing consists on a new matching function, specific for custom properties, and a mechanism to transfer/copy the property’s data to avoid resolving the variable again, since the property’s declaration hasn’t change. We would need to apply the css cascade again, but at least we could save the cost of the variable resolution.

Of course, at the end of the day, improving performance has costs and challenges – and it’s hard to keep performance even once you get it. Bit if we really want performant CSS Custom Properties, this means that we have to decide to prioritize this work. Currently there is reluctance to explore the concept of a new Custom Properties specific cache – the challenge is big and the risks are not non-existent; cache invalidation can get complicated. But, the point is that we have to understand that we aren’t all going to agree what is important enough to warrant attention, or how much investment, or when. Web authors must convince vendors that these use cases are worth being optimized and that the cost and risks of such a complex challenges should be assumed by them.

This work has been sponsored by Bloomberg, which I consider one of the most important contributors of the Web Platform. After several years, the vision of this company and its responsibility as consumer of the platform has lead to many and important contributions that we all enjoy now. Although CSS Grid Layout might be the most remarkable one, there are may other not that big, like this work on CSS Custom Properties, or several other new features of the CSS Text specification. This is a perfect example of an company that tries to change priorities and adapt the web platform to its needs and the use cases they consider more aligned with their business strategy.

I understand that not every user of the web platform can do this kind of investment. This is why I believe that initiatives like Open Priorization could help to move the web platform in a positive direction. By providing a way for us to move past a lot of these conversation and focus on the needs that some web authors and users of the platform consider more important, or higher priority. Improving performance for CSS Custom Properties isn’t currently one of the projects we’ve listed, but perhaps it would be an interesting one we might try in the future if we are successful with these. If you haven’t already, have a look and see if there is something there that is interesting to you or your company – pledges of any size are good – ten thousand $1 donations are every bit as good as ten $1000 donations. Together, we can make a difference, and we all benefit.

Also, we would love to hear about your ideas. Is improving CSS Custom Properties performance important to you? What else is? Share your comments with us on Twitter, either me (@lajava77) or our developer advocate Brian Kardell (@briankardell), or email me at jfernandez@igalia.com. I’d be glad to answer any question about the Open Priorization experiment.

by jfernandez at August 13, 2020 06:16 PM



July 29, 2020

WebKitGTK and WPE WebKit Security Advisory WSA-2020-0007

by The WebKitGTK Project

Several vulnerabilities were discovered in WebKitGTK and WPE WebKit.

  • CVE-2020-9862
    • Versions affected: WebKitGTK before 2.28.4 and WPE WebKit before 2.28.4.
    • Credit to Ophir Lojkine (@lovasoa).
    • Impact: Copying a URL from Web Inspector may lead to command injection. Description: A command injection issue existed in Web Inspector. This issue was addressed with improved escaping.
  • CVE-2020-9893
    • Versions affected: WebKitGTK before 2.28.4 and WPE WebKit before 2.28.4.
    • Credit to 0011 working with Trend Micro Zero Day Initiative.
    • Impact: A remote attacker may be able to cause unexpected application termination or arbitrary code execution. Description: An use-after-free issue was addressed with improved memory management.
  • CVE-2020-9894
    • Versions affected: WebKitGTK before 2.28.4 and WPE WebKit before 2.28.4.
    • Credit to 0011 working with Trend Micro Zero Day Initiative.
    • Impact: A remote attacker may be able to cause unexpected application termination or arbitrary code execution. Description: An out-of-bounds read was addressed with improved input validation.
  • CVE-2020-9895
    • Versions affected: WebKitGTK before 2.28.4 and WPE WebKit before 2.28.4.
    • Credit to Wen Xu of SSLab, Georgia Tech.
    • Impact: A remote attacker may be able to cause unexpected application termination or arbitrary code execution. Description: An use-after-free issue was addressed with improved memory management.
  • CVE-2020-9915
    • Versions affected: WebKitGTK before 2.28.4 and WPE WebKit before 2.28.4.
    • Credit to Ayoub AIT ELMOKHTAR of Noon.
    • Impact: Processing maliciously crafted web content may prevent Content Security Policy from being enforced. Description: An access issue existed in Content Security Policy. This issue was addressed with improved access restrictions.
  • CVE-2020-9925
    • Versions affected: WebKitGTK before 2.28.4 and WPE WebKit before 2.28.4.
    • Credit to an anonymous researcher.
    • Impact: Processing maliciously crafted web content may lead to universal cross site scripting. Description: A logic issue was addressed with improved state management.

We recommend updating to the latest stable versions of WebKitGTK and WPE WebKit. It is the best way to ensure that you are running safe versions of WebKit. Please check our websites for information about the latest stable releases.

Further information about WebKitGTK and WPE WebKit security advisories can be found at: https://webkitgtk.org/security.html or https://wpewebkit.org/security/.

July 29, 2020 12:00 AM



WebKitGTK 2.29.4 released!

by The WebKitGTK Project

This is a development release leading toward 2.30 series.

What’s new in the WebKitGTK 2.29.4 release?

  • Add support for backdrop filters.
  • Add support for text-underline-offset and text-decoration-thickness.
  • Add OpenCDM and AV1 support to media backend.
  • Add new API to get ITP data summary.
  • Use mobile user-agent on tablets.
  • Fix several crashes and rendering issues.

Thanks to all the contributors who made possible this release.

July 29, 2020 12:00 AM



July 28, 2020

WebKitGTK 2.28.4 released!

by The WebKitGTK Project

This is a bug fix release in the stable 2.28 series.

What’s new in the WebKitGTK 2.28.4 release?

  • Fix several crashes and rendering issues.

Thanks to all the contributors who made possible this release.

July 28, 2020 12:00 AM



July 10, 2020

WebKitGTK and WPE WebKit Security Advisory WSA-2020-0006

by The WebKitGTK Project

Several vulnerabilities were discovered in WebKitGTK and WPE WebKit.

  • CVE-2020-9802
    • Versions affected: WebKitGTK before 2.28.3 and WPE WebKit before 2.28.3.
    • Credit to Samuel Groß of Google Project Zero.
    • Impact: Processing maliciously crafted web content may lead to arbitrary code execution. Description: A logic issue was addressed with improved restrictions.
  • CVE-2020-9803
    • Versions affected: WebKitGTK before 2.28.3 and WPE WebKit before 2.28.3.
    • Credit to Wen Xu of SSLab at Georgia Tech.
    • Impact: Processing maliciously crafted web content may lead to arbitrary code execution. Description: A memory corruption issue was addressed with improved validation.
  • CVE-2020-9805
    • Versions affected: WebKitGTK before 2.28.3 and WPE WebKit before 2.28.3.
    • Credit to an anonymous researcher.
    • Impact: Processing maliciously crafted web content may lead to universal cross site scripting. Description: A logic issue was addressed with improved restrictions.
  • CVE-2020-9806
    • Versions affected: WebKitGTK before 2.28.3 and WPE WebKit before 2.28.3.
    • Credit to Wen Xu of SSLab at Georgia Tech.
    • Impact: Processing maliciously crafted web content may lead to arbitrary code execution. Description: A memory corruption issue was addressed with improved state management.
  • CVE-2020-9807
    • Versions affected: WebKitGTK before 2.28.3 and WPE WebKit before 2.28.3.
    • Credit to Wen Xu of SSLab at Georgia Tech.
    • Impact: Processing maliciously crafted web content may lead to arbitrary code execution. Description: A memory corruption issue was addressed with improved state management.
  • CVE-2020-9843
    • Versions affected: WebKitGTK before 2.28.3 and WPE WebKit before 2.28.3.
    • Credit to Ryan Pickren (ryanpickren.com).
    • Impact: Processing maliciously crafted web content may lead to a cross site scripting attack. Description: An input validation issue was addressed with improved input validation.
  • CVE-2020-9850
    • Versions affected: WebKitGTK before 2.28.3 and WPE WebKit before 2.28.3.
    • Credit to @jinmo123, @setuid0x0_, and @insu_yun_en of @SSLab_Gatech working with Trend Micro’s Zero Day Initiative.
    • Impact: A remote attacker may be able to cause arbitrary code execution. Description: A logic issue was addressed with improved restrictions.
  • CVE-2020-13753
    • Versions affected: WebKitGTK before 2.28.3 and WPE WebKit before 2.28.3.
    • Credit to Milan Crha at Red Hat.
    • The bubblewrap sandbox of WebKitGTK and WPE WebKit, prior to 2.28.3, failed to properly block access to CLONE_NEWUSER and the TIOCSTI ioctl. CLONE_NEWUSER could potentially be used to confuse xdg- desktop-portal, which allows access outside the sandbox. TIOCSTI can be used to directly execute commands outside the sandbox by writing to the controlling terminal’s input buffer, similar to CVE-2017-5226.

We recommend updating to the latest stable versions of WebKitGTK and WPE WebKit. It is the best way to ensure that you are running safe versions of WebKit. Please check our websites for information about the latest stable releases.

Further information about WebKitGTK and WPE WebKit security advisories can be found at: https://webkitgtk.org/security.html or https://wpewebkit.org/security/.

July 10, 2020 12:00 AM



July 09, 2020

WebKitGTK 2.28.3 released!

by The WebKitGTK Project

This is a bug fix release in the stable 2.28 series.

What’s new in the WebKitGTK 2.28.3 release?

  • Enable kinetic scrolling with async scrolling.
  • Fix web process hangs on large GitHub pages.
  • Bubblewrap sandbox should not attempt to bind empty paths.
  • Fix threading issues in the media player.
  • Fix several crashes and rendering issues.

Thanks to all the contributors who made possible this release.

July 09, 2020 12:00 AM



July 08, 2020

WebKitGTK 2.29.3 released!

by The WebKitGTK Project

This is a development release leading toward 2.30 series.

What’s new in the WebKitGTK 2.29.3 release?

  • Add webkit_authentication_request_get_security_origin.
  • Change the cookies accept policy to always when no-third-party is set and ITP is enabled.
  • Fix web process hangs on large GitHub pages.
  • Bubblewrap sandbox should not attempt to bind empty paths.
  • Add support for sndio to bubblewrap sandbox.
  • Also handle dark themes when the name ends with -Dark.
  • Fix a race condition causing a crash in media player.
  • Fix several crashes and rendering issues.

Thanks to all the contributors who made possible this release.

July 08, 2020 12:00 AM



July 02, 2020

Web-augmented graphics overlay broadcasting with WPE and GStreamer

by Philippe Normand

Graphics overlays are everywhere nowadays in the live video broadcasting industry. In this post I introduce a new demo relying on GStreamer and WPEWebKit to deliver low-latency web-augmented video broadcasts.

Readers of this blog might remember a few posts about WPEWebKit and a GStreamer element we at Igalia worked on …

by Philippe Normand at July 02, 2020 01:00 PM



June 24, 2020

WebKitGTK 2.29.2 released!

by The WebKitGTK Project

This is a development release leading toward 2.30 series.

What’s new in the WebKitGTK 2.29.2 release?

  • Add Intelligent Tracking Prevention (ITP) support.
  • Add support for video formats in img elements.
  • Add API to handle video autoplay policy that now defaults to disallow autoplay videos with audio.
  • Add API to mute a web view.
  • Add API to allow applications to handle the HTTP authentication credential storage.
  • Add a WebKitSetting to set the media content types requiring hardware support.
  • Fix a crash during drag an drop due to a bug introduced in 2.29.1.
  • Do not start page load during animation in back/forward gesture.
  • Fix several crashes and rendering issues.
  • Translation updates: Ukrainian.

Thanks to all the contributors who made possible this release.

June 24, 2020 12:00 AM



June 16, 2020

WebKit Flatpak SDK and gst-build

by Víctor Jáquez

This post is an annex of Phil’s Introducing the WebKit Flatpak SDK. Please make sure to read it, if you haven’t already.

Recapitulating, nowadays WebKitGtk/WPE developers —and their CI infrastructure— are moving towards to Flatpak-based environment for their workflow. This Flatpak-based environment, or Flatpak SDK for short, can be visualized as a software sandboxed-container, which bundles all the dependencies required to compile, run and debug WebKitGtk/WPE.

In a day-by-day work, this approach removes the potential compilation of the world in order to obtain reproducible builds, improving the development and testing work flow.

But what if you are also involved in the development of one dependency?

This is the case of Igalia’s multimedia team where, besides developing the multimedia features for WebKitGtk and WPE, we also participate in the GStreamer development, the framework used for multimedia.

Because of this, in our workflow we usually need to build WebKit with a fix, hack or new feature in GStreamer. Is it possible to add in Flatpak our custom GStreamer build without messing its own GStreamer setup? Yes, it’s possible.

gst-build is a set of scripts in Python which clone GStreamer repositories, compile them and setup an uninstalled environment. This uninstalled environment allows a transient usage of the compiled framework from their build tree, avoiding installation and further mess up with our system.

The WebKit scripts that wraps Flatpak operations are also capable to handle the scripts of gst-build to build GStreamer inside the container, and, when running WebKit’s artifacts, the scripts enable the mentioned uninstalled environment, overloading Flatpak’s GStreamer.

How do we unveil all this magic?

First of all, setup a gst-build installation as it is documented. In this installation is were the GStreamer plumbing is done.

Later, gst-build operations through WebKit compilation scripts are enabled when the environment variable GST_BUILD_PATH is exported. This variable should point to the directory where the gst-build tree is placed.

And that’s all!

But let’s put these words in actual commands. The following workflow assumes that WebKit repository is cloned in ~/WebKit and the gst-build tree is in ~/gst-build (please, excuse my bashisms).

Compiling WebKitGtk with symbols, using LLVM as toolchain (this command will also compile GStreamer):

$ cd ~/WebKit
% CC=clang CXX=clang++ GST_BUILD_PATH=/home/vjaquez/gst-build Tools/Scripts/build-webkit --gtk --debug
...

Running the generated minibrowser (remind GST_BUILD_PATH is required again for a correct linking):

$ GST_BUILD_PATH=/home/vjaquez/gst-build Tools/Scripts/run-minibrowser --gtk --debug
...

Running media layout tests:

$ GST_BUILD_PATH=/home/vjaquez/gst-build ./Tools/Scripts/run-webkit-tests --gtk --debug media

But wait! There’s more...

What if you I want to parametrize the GStreamer compilation. To say, I would like to enable a GStreamer module or disable the built of a specific element.

gst-build, as the rest of GStreamer modules, uses meson build system, so it’s possible to pass arguments to meson through the environment variable GST_BUILD_ARGS.

For example, I would like to enable gstreamer-vaapi 😇

$ cd ~/WebKit
% CC=clang CXX=clang++ GST_BUILD_PATH=/home/vjaquez/gst-build GST_BUILD_ARGS="-Dvaapi=enabled" Tools/Scripts/build-webkit --gtk --debug
...

by vjaquez at June 16, 2020 11:49 AM



June 09, 2020

WebKitGTK and WPE now supporting videos in the img tag

by Philippe Normand

Using videos in the <img> HTML tag can lead to more responsive web-page loads in most cases. Colin Bendell blogged about this topic, make sure to read his post on the cloudinary website. As it turns out, this feature has been supported for more than 2 years in Safari, but …

by Philippe Normand at June 09, 2020 04:00 PM



June 08, 2020

Introducing the WebKit Flatpak SDK

by Philippe Normand

Working on a web-engine often requires a complex build infrastructure. This post documents our transition from JHBuild to Flatpak for the WebKitGTK and WPEWebKit development builds.

For the last 10 years, WebKitGTK has been relying on a custom JHBuild moduleset to handle its dependencies and (try to) ensure a reproducible …

by Philippe Normand at June 08, 2020 04:50 PM



May 18, 2020

WebKitGTK 2.29.1 released!

by The WebKitGTK Project

This is the first development release leading toward 2.30 series.

What’s new in the WebKitGTK 2.29.1 release?

  • Stop using GTK theming to render form controls.
  • Add API to disable GTK theming for scrollbars too.
  • Fix several race conditions and threading issues in the media player.
  • Add USER_AGENT_BRANDING build option.
  • Add paste as plain text option to the context menu for rich editable content.
  • Fix several crashes and rendering issues.

Thanks to all the contributors who made possible this release.

May 18, 2020 12:00 AM



April 27, 2020

WebKitGTK and WPE WebKit Security Advisory WSA-2020-0005

by The WebKitGTK Project

Several vulnerabilities were discovered in WebKitGTK and WPE WebKit.

  • CVE-2020-3885
    • Versions affected: WebKitGTK before 2.28.0 and WPE WebKit before 2.28.0.
    • Credit to Ryan Pickren (ryanpickren.com).
    • Impact: A file URL may be incorrectly processed. Description: A logic issue was addressed with improved restrictions.
  • CVE-2020-3894
    • Versions affected: WebKitGTK before 2.28.0 and WPE WebKit before 2.28.0.
    • Credit to Sergei Glazunov of Google Project Zero.
    • Impact: An application may be able to read restricted memory. Description: A race condition was addressed with additional validation.
  • CVE-2020-3895
    • Versions affected: WebKitGTK before 2.28.0 and WPE WebKit before 2.28.0.
    • Credit to grigoritchy.
    • Impact: Processing maliciously crafted web content may lead to arbitrary code execution. Description: A memory corruption issue was addressed with improved memory handling.
  • CVE-2020-3897
    • Versions affected: WebKitGTK before 2.28.0 and WPE WebKit before 2.28.0.
    • Credit to Brendan Draper (@6r3nd4n) working with Trend Micro’s Zero Day Initiative.
    • Impact: A remote attacker may be able to cause arbitrary code execution. Description: A type confusion issue was addressed with improved memory handling.
  • CVE-2020-3899
    • Versions affected: WebKitGTK before 2.28.2 and WPE WebKit before 2.28.2.
    • Credit to OSS-Fuzz.
    • Impact: A remote attacker may be able to cause arbitrary code execution. Description: A memory consumption issue was addressed with improved memory handling.
  • CVE-2020-3900
    • Versions affected: WebKitGTK before 2.28.0 and WPE WebKit before 2.28.0.
    • Credit to Dongzhuo Zhao working with ADLab of Venustech.
    • Impact: Processing maliciously crafted web content may lead to arbitrary code execution. Description: A memory corruption issue was addressed with improved memory handling.
  • CVE-2020-3901
    • Versions affected: WebKitGTK before 2.28.0 and WPE WebKit before 2.28.0.
    • Credit to Benjamin Randazzo (@____benjamin).
    • Impact: Processing maliciously crafted web content may lead to arbitrary code execution. Description: A type confusion issue was addressed with improved memory handling.
  • CVE-2020-3902
    • Versions affected: WebKitGTK before 2.28.0 and WPE WebKit before 2.28.0.
    • Credit to Yiğit Can YILMAZ (@yilmazcanyigit).
    • Impact: Processing maliciously crafted web content may lead to a cross site scripting attack. Description: An input validation issue was addressed with improved input validation.

We recommend updating to the latest stable versions of WebKitGTK and WPE WebKit. It is the best way to ensure that you are running safe versions of WebKit. Please check our websites for information about the latest stable releases.

Further information about WebKitGTK and WPE WebKit security advisories can be found at: https://webkitgtk.org/security.html or https://wpewebkit.org/security/.

April 27, 2020 12:00 AM



April 24, 2020

WebKitGTK 2.28.2 released!

by The WebKitGTK Project

This is a bug fix release in the stable 2.28 series.

What’s new in the WebKitGTK 2.28.2 release?

  • Fix excessive CPU usage due to GdkFrameClock not being stopped.
  • Fix UI process crash when EGL_WL_bind_wayland_display extension is not available.
  • Fix position of select popup menus in X11.
  • Fix playing of Youtube ‘live stream’/H264 URLs.
  • Fix a crash under X11 when cairo uses xcb.
  • Fix the build in MIPS64.
  • Fix several crashes and rendering issues.

Thanks to all the contributors who made possible this release.

April 24, 2020 12:00 AM



April 16, 2020

WebKitGTK and WPE WebKit Security Advisory WSA-2020-0004

by The WebKitGTK Project

  • Date Reported: April 16, 2020

  • Advisory ID: WSA-2020-0004

  • CVE identifiers: CVE-2020-11793.

Several vulnerabilities were discovered in WebKitGTK and WPE WebKit.

  • CVE-2020-11793
    • Versions affected: WebKitGTK before 2.28.1 and WPE WebKit before 2.28.1.
    • Credit to Cim Stordal of Cognite.
    • Impact: Processing maliciously crafted web content may lead to arbitrary code execution or application crash (denial of service). Description: A memory corruption issue (use-after-free) was addressed with improved memory handling.

We recommend updating to the latest stable versions of WebKitGTK and WPE WebKit. It is the best way to ensure that you are running safe versions of WebKit. Please check our websites for information about the latest stable releases.

Further information about WebKitGTK and WPE WebKit security advisories can be found at: https://webkitgtk.org/security.html or https://wpewebkit.org/security/.

April 16, 2020 12:00 AM



April 13, 2020

WebKitGTK 2.28.1 released!

by The WebKitGTK Project

This is the first bug fix release in the stable 2.28 series.

What’s new in the WebKitGTK 2.28.1 release?

  • Fix position of default option element popup windows under Wayland.
  • Fix rendering after a cross site navigation with PSON enabled and hardware acceleration forced.
  • Fix a crash in nested wayland compositor when closing a tab with PSON enabled.
  • Update Chrome and Firefox versions in user agent quirks.
  • Fix a crash with bubblewrap sandbox enabled.
  • Fix a crash in JavaScriptCore in ppc64el.
  • Fix the build with GStreamer 1.12.
  • Fix several crashes and rendering issues.

Thanks to all the contributors who made possible this release.

April 13, 2020 12:00 AM



March 31, 2020

Sandboxing WebKitGTK Apps

by Michael Catanzaro

When you connect to a Wi-Fi network, that network might block your access to the wider internet until you’ve signed into the network’s captive portal page. An untrusted network can disrupt your connection at any time by blocking secure requests and replacing the content of insecure requests with its login page. (Of course this can be done on wired networks as well, but in practice it mainly happens on Wi-Fi.) To detect a captive portal, NetworkManager sends a request to a special test address (e.g. http://fedoraproject.org/static/hotspot.txt) and checks to see whether it the content has been replaced. If so, GNOME Shell will open a little WebKitGTK browser window to display http://nmcheck.gnome.org, which, due to the captive portal, will be hijacked by your hotel or airport or whatever to display the portal login page. Rephrased in security lingo: an untrusted network may cause GNOME Shell to load arbitrary web content whenever it wants. If that doesn’t immediately sound dangerous to you, let’s ask me from four years ago why that might be bad:

Web engines are full of security vulnerabilities, like buffer overflows and use-after-frees. The details don’t matter; what’s important is that skilled attackers can turn these vulnerabilities into exploits, using carefully-crafted HTML to gain total control of your user account on your computer (or your phone). They can then install malware, read all the files in your home directory, use your computer in a botnet to attack websites, and do basically whatever they want with it.

If the web engine is sandboxed, then a second type of attack, called a sandbox escape, is needed. This makes it dramatically more difficult to exploit vulnerabilities.

The captive portal helper will pop up and load arbitrary web content without user interaction, so there’s nothing you as a user could possibly do about it. This makes it a tempting target for attackers, so we want to ensure that users are safe in the absence of a sandbox escape. Accordingly, beginning with GNOME 3.36, the captive portal helper is now sandboxed.

How did we do it? With basically one line of code (plus a check to ensure the WebKitGTK version is new enough). To sandbox any WebKitGTK app, just call webkit_web_context_set_sandbox_enabled(). Ta-da, your application is now magically secure!

No, really, that’s all you need to do. So if it’s that simple, why isn’t the sandbox enabled by default? It can break applications that use WebKitWebExtension to run custom code in the sandboxed web process, so you’ll need to test to ensure that your application still works properly after enabling the sandbox. (The WebKitGTK sandbox will become mandatory in the future when porting applications to GTK 4. That’s thinking far ahead, though, because GTK 4 isn’t supported yet at all.) You may need to use webkit_web_context_add_path_to_sandbox() to give your web extension access to directories that would otherwise be blocked by the sandbox.

The sandbox is critically important for web browsers and email clients, which are constantly displaying untrusted web content. But really, every app should enable it. Fix your apps! Then thank Patrick Griffis from Igalia for developing WebKitGTK’s sandbox, and the bubblewrap, Flatpak, and xdg-desktop-portal developers for providing the groundwork that makes it all possible.

by Michael Catanzaro at March 31, 2020 03:56 PM



March 16, 2020

Review of the Igalia Multimedia team Activities (2019/H2)

by Víctor Jáquez

This blog post is a review of the various activities the Igalia Multimedia team was involved along the second half of 2019.

Here are the previous 2018/H2 and 2019/H1 reports.

GstWPE

Succinctly, GstWPE is a GStreamer plugin which allows to render web-pages as a video stream where it frames are GL textures.

Phil, its main author, wrote a blog post explaning at detail what is GstWPE and its possible use-cases. He wrote a demo too, which grabs and previews a live stream from a webcam session and blends it with an overlay from wpesrc, which displays HTML content. This composited live stream can be broadcasted through YouTube or Twitch.

These concepts are better explained by Phil himself in the following lighting talk, presented at the last GStreamer Conference in Lyon:

Video Editing

After implementing a deep integration of the GStreamer Editing Services (a.k.a GES) into Pixar’s OpenTimelineIO during the first half of 2019, we decided to implement an important missing feature for the professional video editing industry: nested timelines.

Toward that goal, Thibault worked with the GSoC student Swayamjeet Swain to implement a flexible API to support nested timelines in GES. This means that users of GES can now decouple each scene into different projects when editing long videos. This work is going to be released in the upcoming GStreamer 1.18 version.

Henry Wilkes also implemented the support for nested timeline in OpenTimelineIO making GES integration one of the most advanced one as you can see on that table:

Feature OTIO EDL FCP7 XML FCP X AAF RV ALE GES
Single Track of Clips ✔ ✔ ✔ ✔ ✔ W-O ✔ ✔
Multiple Video Tracks ✔ ✖ ✔ ✔ ✔ W-O ✔ ✔
Audio Tracks & Clips ✔ ✔ ✔ ✔ ✔ W-O ✔ ✔
Gap/Filler ✔ ✔ ✔ ✔ ✔ ✔ ✖ ✔
Markers ✔ ✔ ✔ ✔ ✖ N/A ✖ ✔
Nesting ✔ ✖ ✔ ✔ ✔ W-O ✔ ✔
Transitions ✔ ✔ ✖ ✖ ✔ W-O ✖ ✔
Audio/Video Effects ✖ ✖ ✖ ✖ ✖ N/A ✖ ✔
Linear Speed Effects ✔ ✔ ✖ ✖ R-O ✖ ✖ ✖
Fancy Speed Effects ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖
Color Decision List ✔ ✔ ✖ ✖ ✖ ✖ N/A ✖

Along these lines, Thibault delivered a 15 minutes talk, also in the GStreamer Conference 2019:

After detecting a few regressions and issues in GStreamer, related to frame accuracy, we decided to make sure that we can seek in a perfectly frame accurate way using GStreamer and the GStreamer Editing Services. In order to ensure that, an extensive integration testsuite has been developed, mostly targeting most important container formats and codecs (namely mxf, quicktime, h264, h265, prores, jpeg) and issues have been fixed in different places. On top of that, new APIs are being added to GES to allow expressing times in frame number instead of nanoseconds. This work is still ongoing but should be merged in time for GStreamer 1.18.

GStreamer Validate Flow

GstValidate has been turning into one of the most important GStreamer testing tools to check that elements behave as they are supposed to do in the framework.

Along with our MSE work, we found that other way to specify tests, related with produced buffers and events through specific pads, was needed. Thus, Alicia developed a new plugin for GstValidate: Validate Flow.

Alicia gave an informative 30 minutes talk about GstValidate and the new plugin in the last GStreamer Conference too:

GStreamer VAAPI

Most of the work along the second half of 2019 were maintenance tasks and code reviews.

We worked mainly on memory restrictions per backend driver, and we reviewed a big refactor: internal encoders now use GstObject, instead of the custom GstVaapiObject. Also we reviewed patches for new features such as video rotation and cropping in vaapipostproc.

Servo multimedia

Last year we worked integrating media playing in Servo. We finally delivered hardware accelerated video playback in Linux and Android. We worked also for Windows and Mac ports but they were not finished. As natural, most of the work were in servo/media crate, pushing code and reviewing contributions. The major tasks were to rewrite the media player example and the internal source element looking to handle the download playbin‘s flag properly.

We also added WebGL integration support with <video> elements, thus webpages can use video frames as WebGL textures.

Finally we explored how to isolate the multimedia processing in a dedicated thread or process, but that task remains pending.

WebKit Media Source Extension

We did a lot of downstream and upstream bug fixing and patch review, both in WebKit and GStreamer, for our MSE GStreamer-based backend.

Along this line we improved WebKitMediaSource to use playbin3 but also compatibility with older GStreamer versions was added.

WebKit WebRTC

Most of the work in this area were maintenance and fix regressions uncovered by the layout tests. Besides, the support for the Rasberry Pi was improved by handling encoded streams from v4l2 video sources, with some explorations with Minnowboard on top of that.

Conferences

GStreamer Conference

Igalia was Gold sponsor this last GStreamer Conference held in Lyon, France.

All team attended and five talks were delivered. Only Thibault presented, besides the video editing one which we already referred, another two more: One about GstTranscoder API and the other about the new documentation infrastructure based in Hotdoc:

We also had a productive hackfest, after the conference, where we worked on AV1 Rust decoder, HLS Rust demuxer, hardware decoder flag in playbin, and other stuff.

Linaro Connect

Phil attended the Linaro Connect conference in San Diego, USA. He delivered a talk about WPE/Multimedia which you can enjoy here:

Demuxed

Charlie attended Demuxed, in San Francisco. The conference is heavily focused on streaming and codec engineering and validation. Sadly there are not much interest in GStreamer, as the main focus is on FFmpeg.

RustFest

Phil and I attended the last RustFest in Barcelona. Basically we went to meet with the Rust community and we attended the “WebRTC with GStreamer-rs” workshop presented by Sebastian Dröge.

by vjaquez at March 16, 2020 03:20 PM



March 12, 2020

WebKitGTK and WPE WebKit Security Advisory WSA-2020-0003

by The WebKitGTK Project

  • Date Reported: March 12, 2020

  • Advisory ID: WSA-2020-0003

  • CVE identifiers: CVE-2020-10018.

Several vulnerabilities were discovered in WebKitGTK and WPE WebKit.

  • CVE-2020-10018
    • Versions affected: WebKitGTK before 2.28.0 and WPE WebKit before 2.28.0.
    • Credit to Sudhakar Verma, Ashfaq Ansari & Siddhant Badhe - Project Srishti of CloudFuzz.
    • Impact: Processing maliciously crafted web content may lead to arbitrary code execution. Description: A memory corruption issue (use-after-free) was addressed with improved memory handling.

We recommend updating to the latest stable versions of WebKitGTK and WPE WebKit. It is the best way to ensure that you are running safe versions of WebKit. Please check our websites for information about the latest stable releases.

Further information about WebKitGTK and WPE WebKit security advisories can be found at: https://webkitgtk.org/security.html or https://wpewebkit.org/security/.

March 12, 2020 12:00 AM



March 11, 2020

Epiphany 3.36 and WebKitGTK 2.28

by Michael Catanzaro

So, what’s new in Epiphany 3.36?

PDF.js

Once upon a time, beginning with GNOME 3.14, Epiphany had supported displaying PDF documents via the Evince NPAPI browser plugin developed by Carlos Garcia Campos. Unfortunately, because NPAPI plugins have to use X11-specific APIs to draw web content, this didn’t  suffice for very long. When GNOME switched to Wayland by default in GNOME 3.24 (yes, that was three years ago!), this functionality was left behind. Using an NPAPI plugin also meant the code was inherently unsandboxable and tied to a deprecated technology. Epiphany disabled support for NPAPI plugins by default in Epiphany 3.30, hiding the functionality behind a hidden setting, which has now finally been removed for Epiphany 3.36, killing off NPAPI for good.

Jan-Michael Brummer, who comaintains Epiphany with me, tried bringing back PDF support for Epiphany 3.34 using libevince, but eventually we decided to give up on this approach due to difficulty solving some user experience issues. Also, the rendering occurred in the unsandboxed UI process, which was again not good for security.

But PDF support is now back in Epiphany 3.36, and much better than before! Thanks to Jan-Michael, Epiphany now supports displaying PDFs using the amazing PDF.js. We are thankful for Mozilla’s work in developing PDF.js and open sourcing it for us to use. Viewing PDFs in Epiphany using PDF.js is more convenient than downloading them and opening them in Evince, and because the PDF is rendered in the sandboxed web process, using web technologies rather than poppler, it’s also approximately one bazillion times more secure.

Screenshot of Epiphany displaying a PDF document
Look, it’s a PDF!

One limitation of PDF.js is that it does not support forms. If you need to fill out PDF forms, you’ll need to download the PDF and open it in Evince, just as you would if using Firefox.

Dark Mode

Thanks to Carlos Garcia, it should finally be possible to use Epiphany with dark GTK themes. WebKitGTK has historically rendered HTML elements using the GTK theme, which has not been good for users of dark themes, which broke badly on many websites, usually due to dark text being drawn on dark backgrounds or various other problems with unexpected dark widgets. Since WebKitGTK 2.28, WebKit will try to manually change to a light GTK theme when it thinks a dark theme is in use, then use the light theme to render web content. (This work has actually been backported to WebKitGTK 2.26.4, so you don’t need to upgrade to WebKitGTK 2.28 to benefit, but the work landed very recently and we haven’t blogged about it yet.) Thanks to Cassidy James from elementary for providing example pages for testing dark mode behavior.

Screenshot demonstrating broken dark mode support
Broken dark mode support prior to WebKitGTK 2.26.4. Notice that the first two pages use dark color schemes when light color schemes are expected, and the dark blue links are hard to read over the dark gray background. Also notice that the text in the second image is unreadable.
Screenshot demonstrating fixed dark mode support in WebKitGTK 2.26.4
Since WebKitGTK 2.26.4, dark mode works as it does in most other browsers. Websites that don’t support dark mode are light, and websites that do support dark mode are dark. Widgets themed using GTK are always light.

Since Carlos had already added support for the prefers-color-scheme media query last year, this now gets us up to dark mode parity with most browsers, except, notably, Safari. Unlike other browsers, Safari allows websites to opt-in to rendering dark system widgets, like WebKitGTK used to do before these changes. Whether to support this in WebKitGTK remains to-be-determined.

Process Swap on Navigation (PSON)

PSON, which debuted in Safari 13, is a major change in WebKit’s process model. PSON is the first component of site isolation, which Chrome has supported for some time, and which Firefox is currently working towards. If you care about web security, you should care a lot about site isolation, because the web browser community has arrived at a consensus that this is the best way to mitigate speculative execution attacks.

Nowadays, all modern web browsers use separate, sandboxed helper processes to render web content, ensuring that the main user interface process, which is unsandboxed, does not touch untrusted web content. Prior to 3.36, Epiphany already used a separate web process to display each browser tab (except for “related views,” where one tab opens another and gains scripting ability over the opened tab, subject to the Same Origin Policy). But in Epiphany 3.36, we now also have a separate web process per website. Each tab will swap between different web processes when navigating between different websites, to prevent any one web process from loading content from different websites.

To make these process swap navigations fast, a pool of prewarmed processes is used to hide the startup cost of launching a new process by ensuring the new process exists before it’s needed; otherwise, the overhead of launching a new web process to perform the navigation would become noticeable. And suspended processes live on after they’re no longer in use because they may be needed for back/forward navigations, which use WebKit’s page cache when possible. (In the page cache, pages are kept in memory indefinitely, to make back/forward navigations fast.)

Due to internal refactoring, PSON previously necessitated some API breakage in WebKitGTK 2.26 that affected Evolution and Geary: WebKitGTK 2.26 deprecated WebKit’s single web process model and required that all applications use one web process per web view, which Evolution and Geary were not, at the time, prepared to handle. We tried hard to avoid this, because we hate to make behavioral changes that break applications, but in this case we decided it was unavoidable. That was the status quo in 2.26, without PSON, which we disabled just before releasing 2.26 in order to limit application breakage to just Evolution and Geary. Now, in WebKitGTK 2.28, PSON is finally available for applications to use on an opt-in basis. (It will become mandatory in the future, for GTK 4 applications.) Epiphany 3.36 opts in. To make this work, Carlos Garcia designed new WebKitGTK APIs for cross-process communication, and used them to replace the private D-Bus server that Epiphany previously used for this purpose.

WebKit still has a long way to go to fully implement site isolation, but PSON is a major step down that road. Thanks to Brady Eidson and Chris Dumez from Apple for making this work, and to Carlos Garcia for handling most of the breakage (there was a lot). As with any major intrusive change of such magnitude, regressions are inevitable, so don’t hesitate to report issues on WebKit Bugzilla.

highlight.js

Once upon a time, WebKit had its own implementation for viewing page source, but this was removed from WebKit way back in 2014, in WebKitGTK 2.6. Ever since, Epiphany would open your default text editor, usually gedit, to display page source. Suffice to say that this was not a very satisfactory solution.

I finally managed to implement view source mode at the Epiphany level for Epiphany 3.30, but I had trouble making syntax highlighting work. I tried using various open source syntax highlighting libraries, but most are designed to highlight small amounts of code, not large web pages. The libraries I tried were not fast enough, so I gave up on syntax highlighting at the time.

Thanks to Jan-Michael, Epiphany 3.36 supports syntax highlighting using highlight.js, so we finally have view source mode working fully properly once again. It works much better than my failed attempts with different JS libraries. Please thank the highlight.js developers for maintaining this library, and for making it open source.

Screenshot displaying Epiphany's view source mode
Colors!

Service Workers

Service workers are now available in WebKitGTK 2.28. Our friends at Apple had already implemented service worker support a couple years ago for Safari 11, but we were pretty slow in bringing this functionality to Linux. Finally, WebKitGTK should now be up to par with Safari in this regard.

Cookies!

Patrick Griffis has updated libsoup and WebKitGTK to support SameSite cookies. He’s also tightened up our cookie policy by implementing strict secure cookies, which prevents http:// pages from setting secure cookies (as they could overwrite secure cookies set by https:// pages).

Adaptive Design

As usual, there are more adaptive design improvements throughout the browser, to provide a better user experience on the Librem 5. There’s still more work to be done here, but Epiphany continues to provide the best user experience of any Linux browser at small screen sizes. Thanks to Adrien Plazas and Jan-Michael for their continued work on this.

Screenshot showing Epiphany running in mobile mode at small window size.
As before, simply resize your browser window to see Epiphany dynamically transition between desktop mode and mobile mode.

elementary OS

With help from Alexander Mikhaylenko, we’ve also upstreamed many elementary OS design changes, which will be used when running under the Pantheon desktop (and not impact users on other desktops), so that the elementary developers don’t need to maintain their customizations as separate patches anymore. This will eliminate a few elementary-specific bugs, including some keyboard shortcuts that were previously broken only in elementary, and some odd tab bar behavior. Although Epiphany still doesn’t feel quite as native as an app designed just for elementary OS, it’s getting closer.

Epiphany 3.34

I failed to blog about Epiphany 3.34 when I released it last September. Hopefully you have updated to 3.34 already, and are already enjoying the two big features from this release: the new adblocker, and the bubblewrap sandbox.

The new adblocker is based on WebKit Content Blockers, which was developed by Apple several years ago. Adrian Perez developed new WebKitGTK API to expose this functionality, changed Epiphany to use it, and deleted Epiphany’s older resource-hungry adblocker that was originally copied from Midori. Previously, Epiphany kept a large GHashMap of compiled regexes in every web process, consuming a very significant amount of RAM for each process. It also took time to compile these regexes when launching each new web process. Now, the adblock filters are instead compiled into an efficient bytecode format that gets mmapped between all web processes to avoid excessive resource use. The bytecode is interpreted by WebKit itself, rather than by Epiphany’s web process extension (which Epiphany uses to execute custom code in WebKit’s web process), for greatly improved performance.

Lastly, Epiphany 3.34 enabled Patrick’s bubblewrap sandbox, which was added in WebKitGTK 2.26. Bubblewrap is an amazing sandboxing tool, already used effectively by flatpak and rpm-ostree, and I’m very pleased with Patrick’s decision to use it for WebKit as well. Because enabling the sandbox can break applications, it is currently opt-in for GTK 3 apps (but will become mandatory for GTK 4 apps). If your application uses WebKitGTK, you really need to take some time to enable this sandbox using webkit_web_context_set_sandbox_enabled(). The sandbox has introduced a couple regressions that we didn’t notice until too late; notably,  printing no longer works, which, half a year later, we still haven’t managed to fix yet. (I’ll try to get to it soon.)

OK, this concludes your 3.36 and 3.34 updates. Onward to 3.38!

by Michael Catanzaro at March 11, 2020 03:00 PM



March 10, 2020

WebKitGTK 2.28.0 released!

by The WebKitGTK Project

This is the first stable release in the 2.28 series.

Highlights of the WebKitGTK 2.28.0 release

  • Add API to enable Process Swap on (Cross-site) Navigation.
  • Add user messages API for the communication with the web extension.
  • Add support for same-site cookies.
  • Service workers are enabled by default.
  • Add support for Pointer Lock API.
  • Add flatpak sandbox support.
  • Make ondemand hardware acceleration policy never leave accelerated compositing mode.
  • Always use a light theme for rendering form controls.
  • Add about:gpu to show information about the graphics stack.

For more details about all the changes included in WebKitGTK 2.28 see the NEWS file that is included in the tarball.

Thanks to all the contributors who made possible this release.

March 10, 2020 12:00 AM



February 27, 2020

WebKitGTK 2.27.91 released!

by The WebKitGTK Project

This is a development release leading toward 2.28 series.

What’s new in the WebKitGTK 2.27.91 release?

  • Update user agent quirks to fix the unsupported browser message in several google services.
  • Fix several compile warnings with GCC 10.
  • Fix the build with GCC 10.
  • Fix several crashes and rendering issues.
  • Translation updates: Chinese

Thanks to all the contributors who made possible this release.

February 27, 2020 12:00 AM



February 14, 2020

WebKitGTK and WPE WebKit Security Advisory WSA-2020-0002

by The WebKitGTK Project

Several vulnerabilities were discovered in WebKitGTK and WPE WebKit.

  • CVE-2020-3862
    • Versions affected: WebKitGTK before 2.26.4 and WPE WebKit before 2.26.4.
    • Credit to Srikanth Gatta of Google Chrome.
    • Impact: A malicious website may be able to cause a denial of service. Description: A denial of service issue was addressed with improved memory handling.
  • CVE-2020-3864
    • Versions affected: WebKitGTK before 2.26.4 and WPE WebKit before 2.26.4.
    • Credit to Ryan Pickren (ryanpickren.com).
    • Impact: A DOM object context may not have had a unique security origin. Description: A logic issue was addressed with improved validation.
  • CVE-2020-3865
    • Versions affected: WebKitGTK before 2.26.4 and WPE WebKit before 2.26.4.
    • Credit to Ryan Pickren (ryanpickren.com).
    • Impact: A top-level DOM object context may have incorrectly been considered secure. Description: A logic issue was addressed with improved validation.
  • CVE-2020-3867
    • Versions affected: WebKitGTK before 2.26.4 and WPE WebKit before 2.26.4.
    • Credit to an anonymous researcher.
    • Impact: Processing maliciously crafted web content may lead to universal cross site scripting. Description: A logic issue was addressed with improved state management.
  • CVE-2020-3868
    • Versions affected: WebKitGTK before 2.26.4 and WPE WebKit before 2.26.4.
    • Credit to Marcin Towalski of Cisco Talos.
    • Impact: Processing maliciously crafted web content may lead to arbitrary code execution. Description: Multiple memory corruption issues were addressed with improved memory handling.

We recommend updating to the latest stable versions of WebKitGTK and WPE WebKit. It is the best way to ensure that you are running safe versions of WebKit. Please check our websites for information about the latest stable releases.

Further information about WebKitGTK and WPE WebKit security advisories can be found at: https://webkitgtk.org/security.html or https://wpewebkit.org/security/.

February 14, 2020 12:00 AM



WebKitGTK 2.26.4 released!

by The WebKitGTK Project

This is a bug fix release in the stable 2.26 series.

What’s new in the WebKitGTK 2.26.4 release?

  • Always use a light theme for rendering form controls.
  • Fix the build with WPE renderer disabled.
  • Fix the build with OpenGL disabled.
  • Fix the build with GCC 10.
  • Fix several crashes and rendering issues.

Thanks to all the contributors who made possible this release.

February 14, 2020 12:00 AM



February 10, 2020

WebKitGTK 2.27.90 released!

by The WebKitGTK Project

This is a development release leading toward 2.28 series.

What’s new in the WebKitGTK 2.27.90 release?

  • Add support for same-site cookies.
  • Add flatpak sandbox support.
  • Enable WebAudio and WebGL by default in WebKitSettings.
  • Add a setting to disallow top level navigation to a data URI.
  • Add support for the -webkit-font-smoothing CSS property.
  • Always use a light theme for rendering form controls.
  • Stop making the Web Inspector windows transient.
  • Ensure mouse cursor is hidden during fullscreen video playback.
  • Add support for inspecting service workers to the remote inspector.
  • Fix several crashes and rendering issues.

Thanks to all the contributors who made possible this release.

February 10, 2020 12:00 AM



January 23, 2020

WebKitGTK and WPE WebKit Security Advisory WSA-2020-0001

by The WebKitGTK Project

Several vulnerabilities were discovered in WebKitGTK and WPE WebKit.

  • CVE-2019-8835
    • Versions affected: WebKitGTK before 2.26.3 and WPE WebKit before 2.26.3.
    • Credit to Anonymous working with Trend Micro’s Zero Day Initiative, Mike Zhang of Pangu Team.
    • Impact: Processing maliciously crafted web content may lead to arbitrary code execution. Description: Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8844
    • Versions affected: WebKitGTK before 2.26.3 and WPE WebKit before 2.26.3.
    • Credit to William Bowling (@wcbowling).
    • Impact: Processing maliciously crafted web content may lead to arbitrary code execution. Description: Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8846
    • Versions affected: WebKitGTK before 2.26.3 and WPE WebKit before 2.26.3.
    • Credit to Marcin Towalski of Cisco Talos.
    • Impact: Processing maliciously crafted web content may lead to arbitrary code execution. Description: A use after free issue was addressed with improved memory management.

We recommend updating to the latest stable versions of WebKitGTK and WPE WebKit. It is the best way to ensure that you are running safe versions of WebKit. Please check our websites for information about the latest stable releases.

Further information about WebKitGTK and WPE WebKit security advisories can be found at: https://webkitgtk.org/security.html or https://wpewebkit.org/security/.

January 23, 2020 12:00 AM



January 22, 2020

WebKitGTK 2.26.3 released!

by The WebKitGTK Project

This is a bug fix release in the stable 2.26 series.

What’s new in the WebKitGTK 2.26.3 release?

  • Fix issues while trying to play a video on NextCloud.
  • Make sure the GL video sink uses a valid WebKit shared GL context.
  • Fix vertical alignment of text containing arabic diacritics.
  • Fix build with icu 65.1.
  • Fix page loading errors with websites using HSTS.
  • Fix web process crash when displaying a KaTeX formula.
  • Fix several crashes and rendering issues.

Thanks to all the contributors who made possible this release.

January 22, 2020 12:00 AM



January 10, 2020

WebKitGTK 2.27.4 released!

by The WebKitGTK Project

This is a development release leading toward 2.28 series.

What’s new in the WebKitGTK 2.27.4 release?

  • Add API for input methods.
  • Add API to serialize/deserialize a JSCValue to/from a JSON string.
  • Add support for strict secure cookies.
  • Add support for saving data from remote inspector.
  • Make ondemand hardware acceleration policy never leave accelerated compositing mode.
  • Fix rendering of conic gradients in high resolution displays.
  • Fix special combination characters not respecting the keystroke order when high CPU load.
  • Honor the IndexedDB directory set in WebsiteDataManager.
  • Fix rendering of text when there’s an initial advance in the text run.
  • Fix web process crash when displaying a KaTeX formula.
  • Fix network process crash with PSON enabled.
  • Fix several crashes and rendering issues.

Thanks to all the contributors who made possible this release.

January 10, 2020 12:00 AM



December 08, 2019

HTML overlays with GstWPE, the demo

by Philippe Normand

Once again this year I attended the GStreamer conference and just before that, Embedded Linux conference Europe which took place in Lyon (France). Both events were a good opportunity to demo one of the use-cases I have in mind for GstWPE, HTML overlays!

As we, at Igalia, usually have a …

by Philippe Normand at December 08, 2019 02:00 PM



November 26, 2019

WebKitGTK 2.27.3 released!

by The WebKitGTK Project

This is a development release leading toward 2.28 series.

What’s new in the WebKitGTK 2.27.3 release?

  • Add support for Pointer Lock API.
  • Improve performance when falling back to system fonts.
  • Stop using DBus for the remote inspector implementation to improve the performance of both WebDriver and remote inspector.
  • Implement support for new ARIA roles: code, strong, emphasis, generic.
  • Fix handling of content type with new custom protocols implementation.
  • Make image decoders fully thread safe.
  • Add support for get page source command in WebDriver.
  • Add support for network proxy capabilities in WebDriver.
  • Add support for new window command in WebDriver.
  • Fix several crashes and rendering issues.
  • Translation updates: Brazilian Portuguese, Ukrainian.

Thanks to all the contributors who made possible this release.

November 26, 2019 12:00 AM



November 08, 2019

WebKitGTK and WPE WebKit Security Advisory WSA-2019-0006

by The WebKitGTK Project

Several vulnerabilities were discovered in WebKitGTK and WPE WebKit.

  • CVE-2019-8710
    • Versions affected: WebKitGTK before 2.26.0 and WPE WebKit before 2.26.0.
    • Credit to found by OSS-Fuzz.
    • Impact: Processing maliciously crafted web content may lead to arbitrary code execution. Description: Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8743
    • Versions affected: WebKitGTK before 2.26.0 and WPE WebKit before 2.26.0.
    • Credit to zhunki from Codesafe Team of Legendsec at Qi’anxin Group.
    • Impact: Processing maliciously crafted web content may lead to arbitrary code execution. Description: Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8764
    • Versions affected: WebKitGTK before 2.26.0 and WPE WebKit before 2.26.0.
    • Credit to Sergei Glazunov of Google Project Zero.
    • Impact: Processing maliciously crafted web content may lead to universal cross site scripting. Description: A logic issue was addressed with improved state management.
  • CVE-2019-8765
    • Versions affected: WebKitGTK before 2.24.4 and WPE WebKit before 2.24.3.
    • Credit to Samuel Groß of Google Project Zero.
    • Impact: Processing maliciously crafted web content may lead to arbitrary code execution. Description: Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8766
    • Versions affected: WebKitGTK before 2.26.0 and WPE WebKit before 2.26.0.
    • Credit to found by OSS-Fuzz.
    • Impact: Processing maliciously crafted web content may lead to arbitrary code execution. Description: Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8782
    • Versions affected: WebKitGTK before 2.26.0 and WPE WebKit before 2.26.0.
    • Credit to Cheolung Lee of LINE+ Security Team.
    • Impact: Processing maliciously crafted web content may lead to arbitrary code execution. Description: Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8783
    • Versions affected: WebKitGTK before 2.26.1 and WPE WebKit before 2.26.1.
    • Credit to Cheolung Lee of LINE+ Graylab Security Team.
    • Impact: Processing maliciously crafted web content may lead to arbitrary code execution. Description: Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8808
    • Versions affected: WebKitGTK before 2.26.0 and WPE WebKit before 2.26.0.
    • Credit to found by OSS-Fuzz.
    • Impact: Processing maliciously crafted web content may lead to arbitrary code execution. Description: Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8811
    • Versions affected: WebKitGTK before 2.26.1 and WPE WebKit before 2.26.1.
    • Credit to Soyeon Park of SSLab at Georgia Tech.
    • Impact: Processing maliciously crafted web content may lead to arbitrary code execution. Description: Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8812
    • Versions affected: WebKitGTK before 2.26.2 and WPE WebKit before 2.26.2.
    • Credit to an anonymous researcher.
    • Impact: Processing maliciously crafted web content may lead to arbitrary code execution. Description: Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8813
    • Versions affected: WebKitGTK before 2.26.1 and WPE WebKit before 2.26.1.
    • Credit to an anonymous researcher.
    • Impact: Processing maliciously crafted web content may lead to universal cross site scripting. Description: A logic issue was addressed with improved state management.
  • CVE-2019-8814
    • Versions affected: WebKitGTK before 2.26.2 and WPE WebKit before 2.26.2.
    • Credit to Cheolung Lee of LINE+ Security Team.
    • Impact: Processing maliciously crafted web content may lead to arbitrary code execution. Description: Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8815
    • Versions affected: WebKitGTK before 2.26.0 and WPE WebKit before 2.26.0.
    • Credit to Apple.
    • Impact: Processing maliciously crafted web content may lead to arbitrary code execution. Description: Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8816
    • Versions affected: WebKitGTK before 2.26.1 and WPE WebKit before 2.26.1.
    • Credit to Soyeon Park of SSLab at Georgia Tech.
    • Impact: Processing maliciously crafted web content may lead to arbitrary code execution. Description: Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8819
    • Versions affected: WebKitGTK before 2.26.1 and WPE WebKit before 2.26.1.
    • Credit to Cheolung Lee of LINE+ Security Team.
    • Impact: Processing maliciously crafted web content may lead to arbitrary code execution. Description: Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8820
    • Versions affected: WebKitGTK before 2.26.1 and WPE WebKit before 2.26.1.
    • Credit to Samuel Groß of Google Project Zero.
    • Impact: Processing maliciously crafted web content may lead to arbitrary code execution. Description: Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8821
    • Versions affected: WebKitGTK before 2.24.4 and WPE WebKit before 2.24.3.
    • Credit to Sergei Glazunov of Google Project Zero.
    • Impact: Processing maliciously crafted web content may lead to arbitrary code execution. Description: Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8822
    • Versions affected: WebKitGTK before 2.24.4 and WPE WebKit before 2.24.3.
    • Credit to Sergei Glazunov of Google Project Zero.
    • Impact: Processing maliciously crafted web content may lead to arbitrary code execution. Description: Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8823
    • Versions affected: WebKitGTK before 2.26.1 and WPE WebKit before 2.26.1.
    • Credit to Sergei Glazunov of Google Project Zero.
    • Impact: Processing maliciously crafted web content may lead to arbitrary code execution. Description: Multiple memory corruption issues were addressed with improved memory handling.

We recommend updating to the latest stable versions of WebKitGTK and WPE WebKit. It is the best way to ensure that you are running safe versions of WebKit. Please check our websites for information about the latest stable releases.

Further information about WebKitGTK and WPE WebKit security advisories can be found at: https://webkitgtk.org/security.html or https://wpewebkit.org/security/.

November 08, 2019 12:00 AM



November 06, 2019

WebKitGTK 2.26.2 released!

by The WebKitGTK Project

This is a bug fix release in the stable 2.26 series.

What’s new in the WebKitGTK 2.26.2 release?

  • Improve performance of querying system fallback fonts.
  • Don’t use prgname in dbus-proxy socket path.
  • Fix thread-safety issues in image decoders.
  • Fix the build with WebDriver disabled.
  • Disable accelerated compositing when we fail to initialize the EGL dispaly under Wayland.
  • Fill the objects category in emoji picker.
  • Fix several crashes and rendering issues.

Thanks to all the contributors who made possible this release.

November 06, 2019 12:00 AM



October 29, 2019

WebKitGTK and WPE WebKit Security Advisory WSA-2019-0005

by The WebKitGTK Project

Several vulnerabilities were discovered in WebKitGTK and WPE WebKit.

  • CVE-2019-8625
    • Versions affected: WebKitGTK before 2.26.0 and WPE WebKit before 2.26.0.
    • Credit to Sergei Glazunov of Google Project Zero.
    • Impact: Processing maliciously crafted web content may lead to universal cross site scripting. Description: A logic issue was addressed with improved state management.
  • CVE-2019-8674
    • Versions affected: WebKitGTK before 2.24.4 and WPE WebKit before 2.24.3.
    • Credit to Sergei Glazunov of Google Project Zero.
    • Impact: Processing maliciously crafted web content may lead to universal cross site scripting. Description: A logic issue was addressed with improved state management.
  • CVE-2019-8707
    • Versions affected: WebKitGTK before 2.24.4 and WPE WebKit before 2.24.3.
    • Credit to an anonymous researcher working with Trend Micro’s Zero Day Initiative, cc working with Trend Micro Zero Day Initiative.
    • Impact: Processing maliciously crafted web content may lead to arbitrary code execution. Description: Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8719
    • Versions affected: WebKitGTK before 2.24.4 and WPE WebKit before 2.24.3.
    • Credit to Sergei Glazunov of Google Project Zero.
    • Impact: Processing maliciously crafted web content may lead to universal cross site scripting. Description: A logic issue was addressed with improved state management.
  • CVE-2019-8720
    • Versions affected: WebKitGTK before 2.26.0 and WPE WebKit before 2.26.0.
    • Credit to Wen Xu of SSLab at Georgia Tech.
    • Impact: Processing maliciously crafted web content may lead to arbitrary code execution. Description: Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8726
    • Versions affected: WebKitGTK before 2.24.3 and WPE WebKit before 2.24.3.
    • Credit to Jihui Lu of Tencent KeenLab.
    • Impact: Processing maliciously crafted web content may lead to arbitrary code execution. Description: Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8733
    • Versions affected: WebKitGTK before 2.24.4 and WPE WebKit before 2.24.3.
    • Credit to Sergei Glazunov of Google Project Zero.
    • Impact: Processing maliciously crafted web content may lead to arbitrary code execution. Description: Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8735
    • Versions affected: WebKitGTK before 2.24.2 and WPE WebKit before 2.24.2.
    • Credit to G. Geshev working with Trend Micro Zero Day Initiative.
    • Impact: Processing maliciously crafted web content may lead to arbitrary code execution. Description: Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8763
    • Versions affected: WebKitGTK before 2.24.4 and WPE WebKit before 2.24.3.
    • Credit to Sergei Glazunov of Google Project Zero.
    • Impact: Processing maliciously crafted web content may lead to arbitrary code execution. Description: Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8768
    • Versions affected: WebKitGTK before 2.24.0 and WPE WebKit before 2.24.0.
    • Credit to Hugo S. Diaz (coldpointblue).
    • Impact: A user may be unable to delete browsing history items. Description: “Clear History and Website Data” did not clear the history. The issue was addressed with improved data deletion.
  • CVE-2019-8769
    • Versions affected: WebKitGTK before 2.26.0 and WPE WebKit before 2.26.0.
    • Credit to Piérre Reimertz (@reimertz).
    • Impact: Visiting a maliciously crafted website may reveal browsing history. Description: An issue existed in the drawing of web page elements. The issue was addressed with improved logic.
  • CVE-2019-8771
    • Versions affected: WebKitGTK before 2.26.0 and WPE WebKit before 2.26.0.
    • Credit to Eliya Stein of Confiant.
    • Impact: Maliciously crafted web content may violate iframe sandboxing policy. Description: This issue was addressed with improved iframe sandbox enforcement.

We recommend updating to the latest stable versions of WebKitGTK and WPE WebKit. It is the best way to ensure that you are running safe versions of WebKit. Please check our websites for information about the latest stable releases.

Further information about WebKitGTK and WPE WebKit security advisories can be found at: https://webkitgtk.org/security.html or https://wpewebkit.org/security/.

October 29, 2019 12:00 AM



October 22, 2019

WebKitGTK 2.27.2 released!

by The WebKitGTK Project

This is a development release leading toward 2.28 series.

What’s new in the WebKitGTK 2.27.2 release?

  • Add user messages API for the communication with the web extension.
  • Enable service workers by default.
  • Add support for saving data in Web Inspector.
  • More navigation gesture improvement.
  • Fix the build with WebDriver disabled.
  • Show also client EGL extensions in about:gpu.
  • Disable accelerated compositing when we fail to initialize the EGL dispaly under Wayland.
  • Fix several crashes and rendering issues.

Thanks to all the contributors who made possible this release.

October 22, 2019 12:00 AM



October 04, 2019

WebKitGTK 2.27.1 released!

by The WebKitGTK Project

This is the first development release leading toward 2.28 series.

What’s new in the WebKitGTK 2.27.1 release?

  • Enable async scrolling when accelerating compositing policy is ‘always’.
  • Add about:gpu to show information about the graphics stack.
  • Add API to enable Process Swap on (Cross-site) Navigation, that is now disabled by default.
  • Add WebKitWebView:page-id property.
  • Improve swipe navigation gesture style.
  • Fix several crashes and rendering issues.

Thanks to all the contributors who made possible this release.

October 04, 2019 12:00 AM



September 23, 2019

WebKitGTK 2.26.1 released!

by The WebKitGTK Project

This is the first bug fix release in the stable 2.26 series.

What’s new in the WebKitGTK 2.26.1 release?

  • Fix MSE media player with GStreamer 1.14.
  • Fix HTML alternate loads never finishing.
  • Fix web view initialization delay on fisrt load.
  • Validate user agent string set via API.
  • Fix a crash when a web view is destroyed with accelerated compositing mode enabled.
  • Fix EGL initialization with newer versions of Mesa.
  • Do not enable the sandbox inside docker.
  • Fix several crashes and rendering issues.

Thanks to all the contributors who made possible this release.

September 23, 2019 12:00 AM



September 18, 2019

Epiphany Technology Preview Users: Action Required

by Michael Catanzaro

Epiphany Technology Preview has moved from https://sdk.gnome.org to https://nightly.gnome.org. The old Epiphany Technology Preview is now end-of-life. Action is required to update. If you installed Epiphany Technology Preview prior to a couple minutes ago, uninstall it using GNOME Software and then reinstall using this new flatpakref.

Apologies for this disruption.

The main benefit to end users is that you’ll no longer need separate remotes for nightly runtimes and nightly applications, because everything is now hosted in one repo. See Abderrahim’s announcement for full details on why this transition is occurring.

by Michael Catanzaro at September 18, 2019 02:19 PM



September 09, 2019

WebKitGTK 2.26.0 released!

by The WebKitGTK Project

This is the first stable release in the 2.26 series.

Highlights of the WebKitGTK 2.26.0 release

  • Add support for subprocess sandboxing.
  • Add support for HSTS (HTTP Strict Transport Security).
  • Use libwpe with fdo backend to implement accelerated compositing under wayland.
  • Remove support for GTK2 NPAPI plugins.
  • Add support for datalist element in text input fields.
  • Show the emoji chooser popover for editable content.
  • Improve rendering of form controls when GTK theme is dark.
  • Fix rendering artifacts in youtube volume button and github comment box.
  • Single process model has been deprecated for security reasons.

For more details about all the changes included in WebKitGTK 2.26 see the NEWS file that is included in the tarball.

Thanks to all the contributors who made possible this release.

September 09, 2019 12:00 AM



September 08, 2019

WebKit Vulnerabilities Facilitate Human Rights Abuses

by Michael Catanzaro

Chinese state actors have recently abused vulnerabilities in the JavaScriptCore component of WebKit to hack the personal computing devices of Uighur Muslims in the Xinjiang region of China. Mass digital surveillance is a key component of China’s ongoing brutal human rights crackdown in the region.

This has resulted in a public relations drama that is largely a distraction to the issue at hand. Whatever big-company PR departments have to say on the matter, I have no doubt that the developers working on WebKit recognize the severity of this incident and are grateful to Project Zero, which reported these vulnerabilities and has previously provided numerous other high-quality private vulnerability reports. (Many other organizations deserve credit for similar reports, especially Trend Micro’s Zero Day Initiative.)

WebKit as a project will need to reassess certain software development practices that may have facilitated the abuse of these vulnerabilities. The practice of committing security fixes to open source long in advance of corresponding Safari releases may need to be reconsidered.

Sadly, Uighurs should assume their personal computing devices have been compromised by state-sponsored attackers, and that their private communications are not private. Even if not compromised in this particular incident, similar successful attacks are overwhelmingly likely in the future.

by Michael Catanzaro at September 08, 2019 05:32 PM



September 03, 2019

WebKitGTK 2.25.92 released!

by The WebKitGTK Project

This is a development release leading toward 2.26 series.

What’s new in the WebKitGTK 2.25.92 release?

  • Add WEBKIT_USE_SINGLE_WEB_PROCESS environment variable to force single process model in all WebKitWebContext. This is a temporary solution for applications still depending on the single process mode behavior. It will be only available in 2.26 series.
  • Add new API to remove a filter from an user content manager given its identifier.
  • Add support for HSTS.
  • Several improvements and bug fixes in MSE media player.
  • Fix building without unified sources.
  • Fix several crashes and rendering issues.
  • Translation updates: Polish, Ukrainian.

Thanks to all the contributors who made possible this release.

September 03, 2019 12:00 AM



August 29, 2019

WebKitGTK and WPE WebKit Security Advisory WSA-2019-0004

by The WebKitGTK Project

Several vulnerabilities were discovered in WebKitGTK and WPE WebKit.

  • CVE-2019-8644
    • Versions affected: WebKitGTK before 2.24.4 and WPE WebKit before 2.24.3.
    • Credit to G. Geshev working with Trend Micro’s Zero Day Initiative.
    • Processing maliciously crafted web content may lead to arbitrary code execution. Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8649
    • Versions affected: WebKitGTK before 2.24.4 and WPE WebKit before 2.24.3.
    • Credit to Sergei Glazunov of Google Project Zero.
    • Processing maliciously crafted web content may lead to universal cross site scripting. A logic issue existed in the handling of synchronous page loads. This issue was addressed with improved state management.
  • CVE-2019-8658
    • Versions affected: WebKitGTK before 2.24.4 and WPE WebKit before 2.24.3.
    • Credit to akayn working with Trend Micro’s Zero Day Initiative.
    • Processing maliciously crafted web content may lead to universal cross site scripting. A logic issue was addressed with improved state management.
  • CVE-2019-8666
    • Versions affected: WebKitGTK and WPE WebKit before 2.24.3.
    • Credit to Zongming Wang (王宗明) and Zhe Jin (金哲) from Chengdu Security Response Center of Qihoo 360 Technology Co. Ltd.
    • Processing maliciously crafted web content may lead to arbitrary code execution. Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8669
    • Versions affected: WebKitGTK before 2.24.4 and WPE WebKit before 2.24.3.
    • Credit to akayn working with Trend Micro’s Zero Day Initiative.
    • Processing maliciously crafted web content may lead to arbitrary code execution. Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8671
    • Versions affected: WebKitGTK and WPE WebKit before 2.24.2.
    • Credit to Apple.
    • Processing maliciously crafted web content may lead to arbitrary code execution. Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8672
    • Versions affected: WebKitGTK and WPE WebKit before 2.24.2.
    • Credit to Samuel Groß of Google Project Zero.
    • Processing maliciously crafted web content may lead to arbitrary code execution. Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8673
    • Versions affected: WebKitGTK and WPE WebKit before 2.24.3.
    • Credit to Soyeon Park and Wen Xu of SSLab at Georgia Tech.
    • Processing maliciously crafted web content may lead to arbitrary code execution. Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8676
    • Versions affected: WebKitGTK and WPE WebKit before 2.24.3.
    • Credit to Soyeon Park and Wen Xu of SSLab at Georgia Tech.
    • Processing maliciously crafted web content may lead to arbitrary code execution. Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8677
    • Versions affected: WebKitGTK and WPE WebKit before 2.24.2.
    • Credit to Jihui Lu of Tencent KeenLab.
    • Processing maliciously crafted web content may lead to arbitrary code execution. Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8678
    • Versions affected: WebKitGTK before 2.24.4 and WPE WebKit before 2.24.3.
    • Credit to an anonymous researcher, Anthony Lai (@darkfloyd1014) of Knownsec, Ken Wong (@wwkenwong) of VXRL, Jeonghoon Shin (@singi21a) of Theori, Johnny Yu (@straight_blast) of VX Browser Exploitation Group, Chris Chan (@dr4g0nfl4me) of VX Browser Exploitation Group, Phil Mok (@shadyhamsters) of VX Browser Exploitation Group, Alan Ho (@alan_h0) of Knownsec, Byron Wai of VX Browser Exploitation.
    • Processing maliciously crafted web content may lead to arbitrary code execution. Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8679
    • Versions affected: WebKitGTK and WPE WebKit before 2.24.2.
    • Credit to Jihui Lu of Tencent KeenLab.
    • Processing maliciously crafted web content may lead to arbitrary code execution. Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8680
    • Versions affected: WebKitGTK before 2.24.4 and WPE WebKit before 2.24.3.
    • Credit to Jihui Lu of Tencent KeenLab.
    • Processing maliciously crafted web content may lead to arbitrary code execution. Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8681
    • Versions affected: WebKitGTK and WPE WebKit before 2.24.3.
    • Credit to G. Geshev working with Trend Micro Zero Day Initiative.
    • Processing maliciously crafted web content may lead to arbitrary code execution. Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8683
    • Versions affected: WebKitGTK before 2.24.4 and WPE WebKit before 2.24.3.
    • Credit to lokihardt of Google Project Zero.
    • Processing maliciously crafted web content may lead to arbitrary code execution. Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8684
    • Versions affected: WebKitGTK before 2.24.4 and WPE WebKit before 2.24.3.
    • Credit to lokihardt of Google Project Zero.
    • Processing maliciously crafted web content may lead to arbitrary code execution. Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8686
    • Versions affected: WebKitGTK and WPE WebKit before 2.24.2.
    • Credit to G. Geshev working with Trend Micro’s Zero Day Initiative.
    • Processing maliciously crafted web content may lead to arbitrary code execution. Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8687
    • Versions affected: WebKitGTK and WPE WebKit before 2.24.3.
    • Credit to Apple.
    • Processing maliciously crafted web content may lead to arbitrary code execution. Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8688
    • Versions affected: WebKitGTK before 2.24.4 and WPE WebKit before 2.24.3.
    • Credit to Insu Yun of SSLab at Georgia Tech.
    • Processing maliciously crafted web content may lead to arbitrary code execution. Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8689
    • Versions affected: WebKitGTK and WPE WebKit before 2.24.3.
    • Credit to lokihardt of Google Project Zero.
    • Processing maliciously crafted web content may lead to arbitrary code execution. Multiple memory corruption issues were addressed with improved memory handling.
  • CVE-2019-8690
    • Versions affected: WebKitGTK and WPE WebKit before 2.24.3.
    • Credit to Sergei Glazunov of Google Project Zero.
    • Processing maliciously crafted web content may lead to universal cross site scripting. A logic issue existed in the handling of document loads. This issue was addressed with improved state management.

We recommend updating to the latest stable versions of WebKitGTK and WPE WebKit. It is the best way to ensure that you are running safe versions of WebKit. Please check our websites for information about the latest stable releases.

Further information about WebKitGTK and WPE WebKit security advisories can be found at: https://webkitgtk.org/security.html or https://wpewebkit.org/security/.

August 29, 2019 12:00 AM



August 28, 2019

WebKitGTK 2.24.4 released!

by The WebKitGTK Project

This is a bug fix release in the stable 2.24 series.

What’s new in the WebKitGTK 2.24.4 release?

  • Updated the user agent string to make happy certain websites which would claim that the browser being used was unsupported.
  • Improve loading of multimedia streams to avoid memory exhaustion due to excessive caching.
  • Fix display of documents with MIME type application/xml in the Web Inspector, when loaded using XmlHttpRequest.
  • Fix a hang while scrolling certain websites which include HLS video content (Twitter, for example).
  • Fix rounding artifacts in volume levels for media playback.
  • Fix several crashes and rendering issues.
  • Fix the build with video track support disabled.
  • Fix the build with OpenGL support disabled.
  • Fix build issue which would cause media controls to disappear when Python 3.x was used during the build process.

Thanks to all the contributors who made possible this release.

August 28, 2019 12:00 AM