Bettering the Participant on Android. Gray Skold | (former Android Video… | by Pinterest Engineering | Pinterest Engineering Weblog | Dec, 2022

Gray Skold | (former Android Video Engineer) ; Lin Wang | Android Efficiency Engineer; Sheng Liu | Android Efficiency Engineer

Close up of black analog speedometer — Photo by CHUTTERSNAP on Unsplash

Pinterest Android App presents a uncommon expertise with a mixture of photos and movies on a two-column grid. So as to preserve a performant video expertise on Android units, we targeted on:

  • Warming up
  • Configurations
  • Pooling gamers

So as to scale back the startup latency, we set up a video community connection by sending a dummy HTTP HEAD request throughout the early utility startup time. The identical connection can be utilized to play future movies. That is finished even earlier than any video urls are returned from our server.

okhttpNetworkClient.newCall( Request.Builder().url(videoUrl).cacheControl(FORCE_NETWORK).head().build() ).enqueue(callback)

The identical technique additionally applies to UI rendering. We discovered Exoplayer tends to do numerous work after parsing the data from the video url. It:

Since most of our movies’ facet ratios are pre-determined, we are able to stop the above work by:

The latter prevents the participant from attempting to recalculate the facet ratio.

ExoPlayer gives us the setBufferDurationsMs() to customise numerous buffering durations used to delay playback till we have now ample information. Since a majority of Pinterest’s media content material is in short-form, we are able to use a lot shorter buffering durations which is able to lead to much less time ready for information to load.

setBufferDurationsMs( minBufferMs = 1_000, maxBufferMs = 50_000, bufferForPlaybackMs = 500, bufferForPlaybackAfterRebufferMs = 1_000 )

By doing this, we noticed a big discount in video startup latency. Though the rebuffer fee will increase a bit, the general video UX remains to be improved.

Utilizing the next two parameters, we may successfully guarantee movies loaded in-feed are restricted by their viewport dimension (i.e. to keep away from loading a big 1080p video in a small 360 pixel viewport).

buildUponParameters() .setMaxVideoSize(maxVideoWidth, maxVideoHeight) .setExceedVideoConstraintsIfNecessary(false)

The Pinterest app performs a number of muted movies within the grid on the similar time. Utilizing the next parameters, we may disable the audio rendering to avoid wasting community bandwidth to obtain the audio and reminiscence consumption to course of them.

buildUponParameters() .setMaxAudioBitrate(0) .setMaxAudioChannelCount(0) .setExceedAudioConstraintsIfNecessary(false)

Exoplayer gives a cache interface to maintain downloaded media information on disk. Nonetheless, within the circumstances once we run into deadly errors brought on by backend bugs, the unhealthy contents can even stick within the cache. This may trigger the appliance to proceed encountering the identical playback errors despite the fact that the backend is fastened.

We use SimpleCache.removeResource() to purge the soiled cache when the next deadly IO errors are returned from the Player.Listener.onPlayerError():

  • ERROR_CODE_IO_UNSPECIFIED 2000
  • ERROR_CODE_IO_INVALID_HTTP_CONTENT_TYPE 2003
  • ERROR_CODE_IO_BAD_HTTP_STATUS 2004
  • ERROR_CODE_IO_FILE_NOT_FOUND 2005
  • ERROR_CODE_IO_READ_POSITION_OUT_OF_RANGE 2008

Lastly, we constructed our personal cache to pool gamers as they’re wanted. Traditionally, we instantiated new participant situations on the fly, which brought on vital overhead on each the reminiscence and bandwidth. Listed below are some excessive degree learnings:

Separate By Encoding

Participant situations are successfully holding onto a reference to an underlying decoder based mostly on the final rendered media’s encoding sort. It takes a measurable quantity of labor for gamers to modify context between totally different decoders. Subsequently we pooled our gamers based mostly on the preliminary decoding format. By guaranteeing the recycled gamers with decoders matching the media’s encoding, we eliminated any latency overhead brought on when switching encoding codecs.

Good Sizing

The pool dimension itself ran via a number of iterations to search out the perfect area wanted to accommodate a number of video performs whereas avoiding OutOfMemory (OOM) and ANR. The next two APIs are utilized to not over-burdening the system:

OnTrimMemory(int)

This was our key callback used to tell us that we must always clear our participant cache pool as we begin to get dangerously near sending OOMs.

setForegroundMode(true)

Setting this flag will let the Exoplayer to retain a direct reference to the video decoder and preserve it in reminiscence even within the idle state. Nonetheless, it takes a big toll on reminiscence and gadget stability. We needed to construct logic to make use of this technique conservatively, based mostly on each the present utility lifecycle and present accessible reminiscence on the gadget.

Bettering the efficiency of video playback is a endless investigation into the inner-workings of the ExoPlayer library in addition to our personal product’s distinct use-cases, however what’s probably the most difficult is constructing an structure that works for the uniquely Pinterest expertise of the house feed. Our long-term purpose is to share this work to different builders seeking to construct a seamless video UX with minimal setup required. Within the meantime, we hope everybody builds a extra performant video expertise.

To be taught extra about engineering at Pinterest, try the remainder of our Engineering Weblog and go to our Pinterest Labs web site. To discover life at Pinterest, go to our Careers web page.