Creating A Universal Image Crop/Scale Animation for the Web

Background

AMP has an image lightbox component that allows enlarging <img>s within a fullscreen modal. Previously, the lightbox opening/closing animation in AMP could only animate correctly when the <img> had the same aspect ratio in both the page and the lightbox. Additionally, it did not work with values of object-fit other than fill. The object-fit CSS property determines how an image renders within an <img> element. For <img>s where we could not animate, we did a simple fade-in instead.

I wanted to support animating the <img> into lightbox mode, even when the aspect ratios did not match or different values for object-position were used. This would allow a cropped version to appear in the page, with the full version in a lightbox. While implementing this as part of my work on AMP, I made the code a standalone library so developers not using AMP could benefit as well. Below is a video of what I was aiming for.

Note that we are animating the following:

1. The position of the image
2. The size of the image
3. The crop of the image

The position and size of the image are fairly straightforward to animate, but the crop turns out to be more difficult to do in a performant way, which I will get into later.

TL;DR

Implementation Choices

I wanted to ensure that my solution worked well both in AMP and standalone so that others could use it outside of AMP. The code is available on GitHub.

While AMP itself uses Closure Compiler types, I decided to go with TypeScript instead. I was still able to compile using Closure Compiler to generate a minified version with a few extra steps.

Sketching out the API

While constructing the API for my animation function, I wanted to make sure I supported grouping of DOM reads and writes, to avoid forced synchronous layouts. AMP does this internally to ensure components do not spend a lot of time recalculating styles when independent components react to the same events.

To support batching with other reads / writes within AMP, the API needs to be split into two steps. The first step is to measure Elements (for example, where the images are positioned) to figure out the calculations needed to do the animation. The second step is to mutate the DOM (by applying styles) to actually run the animation. The API I came up with looks like:

function prepareImageAnimation(…) {
  … // Measure things, do calculations
  return {
    apply: function() {
      … // apply the DOM mutations
    },
    cleanup: function() {
      … // undo any effects
    },
  };
}

This still allows you to start the animation immediately if you so choose, though it is a bit clumsy for that case. Splitting up the work, allows you to start the calculations earlier (e.g. from touchstart), reducing the latency to actually start the animation. While the code is not computation heavy, the first call can be time consuming since the code to generate the animation needs to be parsed/compiled by the browser. This could potentially take more than a few of milliseconds, especially on lower-end phones.

Preparing to Animate

The first issue that I encountered was that you could crop <img>s using the object-fit CSS property, and there is no way to animate between two different values of this property. As a result, the first step was to display the original <img> in a way that I could control. The structure I came up with looked like:

<div style="
    overflow: hidden;
    display: flex;
    align-items:center;
    justify-content: center">
  <img src="…" style="width: …; height: …;">
</div>

Note that this assumes the rendered image is centered. There is some additional logic needed to handle different values of object-position, which can move the rendered image around within the cropping container.

The code to calculate the rendered image's width/height in an <img> with a given width, height and object-fit value is relatively straightforward. If you are interested, you can take a look here.

Animating Properties

Performing a smooth animation for the rendered image size and position are relatively straightforward using the CSS transform property. It is GPU accelerated and performs well on low end devices, even if the main thread is busy. The crop is, unfortunately, more involved. First, I researched what properties could animated cheaply. I remembered an article from a few years back about which properties were safe to animate. Unfortunately, I found that this has not really changed since, which means you cannot animate clip or the newer clip-path with the same performance.

The basic approach for animating clipping is not too complicated. You can scale up (or down) the container that has overflow: hidden in both the x and y directions to adjust the size. You can then also scale the contents of the container with an inverse scale to counteract the scaling on the container. The result will be a larger clipping container, with content the same size as before.

Note that the curves for the scale up and the counteracting scale are not the same. At any given time, the product of both values should be one. For example, if you have a linear scale up, f(t) = a*t, then you need f(t) = 1/(a*t) to cancel it out as the animation progresses.

I had initially planned to implement the animation for a single timing curve. I started off by using my desired specific cubic-bezier() for the scale animation. I found that for some curves, I could find another curve that counteracted the scaling fairly well, but the starts and ends of the animation could be off enough to be noticeable. That is, the start and end of the animation did not feel as natural as the middle.

I eventually found that for any given input curve, I could approximate the inverting scale using multiple curves, but finding those curves was not trivial. Given this, it made sense to support arbitrary curves instead of a predefined one. This brought me to a few options:

1. Do the animation in JS
2. Do the animation using Animation Worklet
3. Generate keyframes using the Web Animations API
4. Generate keyframes using a dynamically created stylesheet

The first option was a non-starter. I wanted to make sure the animation was smooth no matter what was running on the main thread. The second option is a work in progress API and would only help for Chrome. Since I did not want to require the use of a polyfill for the Web Animations API, the last option won by default.

Generating Keyframes

Keyframes tell the browser the values for different properties at some point within the animation. For example, take the following keyframes:

@keyframes some-name {
  0% { transform: scale(0.1); }
  20% { transform: scale(0.5); }
  100% { transform: scale(1.0); }
}

This tells the browser that it should start the animation at scale 0.1. At 20% into the animation, it should scale to 0.5. Finally, at the end, it should scale to 1.0.

The browser then interpolates between the given keyframes (using the animation-timing-function) to find the scale value at any given time. For example, given:

• The above keyframes
• An animation duration of 1000ms
• A linear animation-timing-function
• An offset of 100ms into the animation

The browser will scale to 0.3 as that is the value half way between the 0% and 20% keyframes.

To animate, I needed to generate two sets of keyframes: one for the scale of the container and one to counteract the scale. The product of these keyframes should always be one (so that the image inside is not affected by the scale for the crop). Note that the interpolated value (from the keyframes) for the scale and counter scale will not exactly multiply to one at an arbitrary point in time. I found I could get it close enough to not be noticeable during the animation, but it does cause other problems. See the lightbox demo section for more info.

In order to generate keyframes, my initial thought was to generate evenly spaced keyframes (e.g. 10%, 20%, ...) to make sure the interpolated value would be close enough at any given time. However, the formula for the cubic-bezier() function does not provide an easy way to get the output value as a function of time. Instead, it lets you calculate x/y pairs given a t parameter. The x value corresponds to a time in the animation, with the y value corresponding to the percentage between the start/end of the animation at that time. This uses the following formula:

The browser's cubic-bezier() function always uses P0 = 0 and P3 = 1. P1 and P2 are the first/second x/y control points passed to the function.

Supporting Scrolling During the Animation

My initial version of the animation used position: fixed on the animating element. As a result, the animation would not behave as expected if the user scrolled during the animation, as illustrated below:

This was corrected by using position: absolute on the animating element. Note, when the target image is within a container that scrolls (other than <body>), the animating element should be placed in that container. For an example, see the hero animation demo.

When using position: absolute, the top and left values need to be set relative to the first ancestor with a position value that is not static (either relative, absolute, sticky or fixed) or the <html> element if no such ancestor exists. This is an element's offsetParent, with two exceptions. First, when there are no positioned ancestors, the offsetParent will be the <body>. When the <body> is not positioned, the <html> element should be used instead. Second, the offsetParent for <body> is null. In this case, the <html> element should be used as well.

Once I had the correct ancestor, I simply needed to calculate the difference between the top/left values for the target <img> and ancestor ClientRects. After making the changes, everything worked as expected:

Supporting object-position

While doing my initial planning / reading for this project, I overlooked the existence of the object-position CSS property. The code I had initially written assumed the rendered image would always be centered. Trying to fix this took some additional time to get right.

While reading how object-position worked was fairly straightforward, I had quite a bit of trouble trying to implement it correctly. The unit tests for the replacement image were not too hard to get working, but the animation itself was still not accurate.

The main problem was that I had not completely removed my assumptions regarding the image's position. Since the scale transforms were using the default transform-origin, which centers as you scale, the image would be rendered with an incorrect position when I went to translate it for the object-position. Changing over to transform-origin: top left made things a lot easier to reason about.

This code ended up adding more complexity than I initially thought. This is reflected in an additional 400 bytes of compressed payload. In order to figure out how the rendered image was positioned inside of <img> Element, I had to parse the value of the object-position property. Fortunately, getComputedStyle() will return resolved values for CSS properties. As a result object-position: center center's computed value is 50% 50%, so I did not need to understand values like center or right.

The computed value can get a bit tricky however, since you can offset the position by a pixel (or other unit) value. For example: object-position: top 2px right 1em, is resolved to object-position: calc(-16px + 100%) 2px. To figure out the percent and pixel values I needed to move things by, I had to write some ad-hoc code, including a regex to extract out the parts. In the future, the CSS Typed Object Model will make getting this information from the browser much easier.

Testing

Ideally, I would have set up screenshot based testing for my code in addition to my other tests. This would normally be done through WebDriver or Puppeteer. I looked a bit into setting this up, but it was not as straightforward as standard unit tests. Thus, I have opted for comparing ClientRects in my tests for now. Since this has made writing the tests cumbersome, I have written fewer tests than I would like.

It was important for me to test how the animation rendered while in progress to make sure all the calculations were correct and that the keyframes were working as expected (and would and would not regress when I changed the code). To do this, I used some code that allows pausing and moving CSS animations to a desired point in time. The code is available on GitHub if you would like to use it for your own tests.

CSS animations can be controlled in testing by first stopping the animation (animation-play-state: paused), and then adding a negative animation-delay to move the animation to the desired position. This adds some bookkeeping to handle existing delays, handle animations on pseudo elements (e.g. ::before) and to handle Shadow DOM, but the code is otherwise fairly simple.

Controlling the animation from the test is pretty easy. The usage in one of my tests looks like:

import {
  setup as setupAnimations,
  tearDown as tearDownAnimations,
  offset,
} from '../testing/animation-test-controller';

describe('prepareImageAnimation', () => {
  // Stops all existing and future animations
  before(() => { setupAnimations(); });
  // Resumes all existing and future animations
  after(() => { tearDownAnimations(); });
  …
  describe('scrolling', () => {
    …
    it('should have the correct position 200ms in', () => {
      // Call the code to setup the animation. It will not actually animate as
      // it is paused.
      startAnimation(largerImg, smallerImg);
      // Moves all the animations to the desired time.
      offset(200);
      window.scrollTo(50, 75);

      // Now I can check how things are positioned and make sure they are
      // correct.
      const replacement = getIntermediateImg();
      const {top, left} = replacement.getBoundingClientRect();
      // 20% of the animation, starting from 10, going to 500 - 75 from scroll
      expect(top).to.be.closeTo(-51.5, COMPARISON_EPSILON);
      // 20% of the animation, starting from 10, going to 100 - 50 from scroll
      expect(left).to.be.closeTo(-37.5, COMPARISON_EPSILON);
    });
  …
  });
});

The tests were helpful in catching problems as I refactored and added features. Failures were hard to debug however. Screenshot based tests would have been more useful when trying to reason about test failures.

Creating Demos

After I completed my initial code, I set out to create a few demos to make sure I had made things flexible enough as well as making sure I did not miss any edge cases.

<div class="gallery">
<div class="thumbnails">
<img srcset="large-1.png 400w, small-1.png 96w" sizes="96w">
<img srcset="large-2.png 400w, small-2.png 96w" sizes="96w">
…
</div>
<div class="primary">
<img srcset="large-1.png 400w, small-1.png 96w" sizes="400w">
</div>
</div>

function loadLargerImgSrc(img, largerSize) {
  …
  const dummyImg = new Image();
  dummyImg.srcset = img.srcset;
  dummyImg.sizes = largerSize;
  …
}

window.expand = function(event) {
  const srcImg = event.target.closest('img');
  const {applyAnimation, cleanupAnimation} = prepareImageAnimation({
    srcImg,
    targetImg,
    …
  });
  // Set the srcset/sizes to use the smaller image we already have.
  targetImg.srcset = srcImg.srcset;
  targetImg.sizes = srcImg.sizes;
  // Preload the larger image before starting the animation.
  loadLargerImgSrc(srcImg, targetImgSizes);
  …
  applyAnimation();
  setTimeout(() => {
    // Setting the original (larger) sizes attribute to use the higher
    // resolution when available.
    targetImg.sizes = targetImgSizes;
    cleanupAnimation();
  }, …);
}

if (srcImg.currentSrc) {
  targetImg.src = srcImg.currentSrc;
  targetImg.srcset = '';
  targetImg.sizes = '';
} else {
  targetImg.src = '';
  targetImg.srcset = srcImg.srcset;
  targetImg.sizes = srcImg.sizes;
}

targetImg.srcset = srcImg.srcset;
targetImg.sizes = targetImgSizes;

How It Went

There was a mixture of things that went well or work well and things that do not.

Conclusion

The web platform is pretty powerful, letting you perform this animation in a performant way even on low end devices. At the same time, it is not always easy to actually implement performant animations. The inline image expansion demo shows how something that feels like it should be easy has a lot of pitfalls in practice. There are also some things that can be a bit tricky to implement and debug without knowing about various odds and ends of CSS.

The web does have a few rough edges still both on the Browser side as well as in tooling. For example, dynamically generating keyframe names is awkward but will be made easier when Web Animations API support becomes more widespread. Tooling still can be a bit of a hassle for more complicated setups (e.g. TypeScript + Closure compiler or screenshot based tests).

Some of the browser APIs can be opaque. I needed to include code to figure out how the image would be rendered given the container size and the object-fit / object-position properties. It would have been nice to be able to get this information from the browser itself. While not a huge amount of extra code for this one case, there are many times where the APIs fall short and we end up writing extra code as developers. These end up adding up to a lot of extra bytes we need to ship down to clients.

Overall, I am really happy with how smooth the animation turned out, even on slower devices. I hope that this sort of polish becomes more common on the web over time and does not remain solely in mobile apps. Please feel free to check out the code, the documentation, and the demos to see how to use it on your own page. It is also available via npm.

Appendix