Dealing with large files in Cordova

Article header
Article header

While it may seem like uploading files in Cordova applications is a trivial task easily completed with cordova-plugin-file-transfer plugin.

But since November 2017 this plugin is deprecated and it is recommended to use XMLHttpRequest to handle this task.

According to Cordova's blog post, we must resolve the file in the local filesystem, read it via FileReader and send it using XMLHttpRequest. But if we try to upload a file that exceeds the size of 30MB (this may vary depending on device) we can get the following console error:

POST https://some/post/url net::ERR_FILE_NOT_FOUND

When we try to execute:

    // Resolve fileLocalUrl to File object
    const resolveFile = (url) => {
      return new Promise((resolve, reject) => {
        window.resolveLocalFileSystemURL(
          url,
          (fileEntry) => {
            fileEntry.file(resolve);
          });
      });
    };

    // Read File's Blob
    const readFile = (file) => {
      return new Promise((resolve, reject) => {
        const reader = new FileReader();
        reader.onloadend = (ev) => {
          if (ev.target.error) {
            return reject(ev.target.error);
          }
          return resolve(new Blob([new Uint8Array(reader.result)]));
        };
        reader.readAsArrayBuffer(file);
      });
    };

    // Get duration of video file
    const getDuration = (blob) => {
      return new Promise((resolve, reject) => {
        const video = document.createElement('video');
        const videoSrc = document.createElement('source');
        video.preload = 'metadata';
        video.appendChild(videoSrc)

        video.onloadedmetadata = videoSrc.onloadedmetadata = () => {
          resolve(Math.round(video.duration));
          if (typeof videoSrc.remove === 'function') {
            videoSrc.remove();
          }
          if (typeof video.remove === 'function') {
            video.remove();
          }
        };
        video.onerror = videoSrc.onerror = reject;

        videoSrc.src = URL.createObjectURL(blob);
        if (typeof video.load === 'function') {
          video.load();
        }
      });
    };

    resolveFile(fileLocalUrl)
      .then(readFile)
      .then(getDuration)
      .then((duration) => {
        console.log('video duration is: ' + duration);
      })
      .catch((err) => {
        console.error(err);
      });

We get an error like this:

GET blob:file:///a06f500c-79b8-4176-8d1a-39bcf8d0b3d4 500 (Internal Server Error)

It appears the problem is with neither XMLHttpRequest, nor URL.createObjectURL.

If we repeat those operations on a small file (< 10MB) – no error appears.

The solution to this problem is surprising. Because WebView has a data transfer limit when working with native code, we need to read the file's blob in chunks. After that, we can join these chunks into a single blob and use it! Here is a replacement readFile function:

    const readFileChunk = (file, start, chunkSize, chunks) {
      return new Promise((resolve, reject) => {
        const reader = new FileReader();
        reader.onloadend = (ev) => {
          if (ev.target.error) {
            return reject(ev.target.error);
          }
          resolve(new Blob([new Uint8Array(reader.result)]));
        }

        let toRead = file;
        if (typeof start !== 'undefined' && typeof chunkSize !== 'undefined') {
          toRead = file.slice(start, start + chunkSize);
          start += chunkSize;
        }

        return reader.readAsArrayBuffer(toRead);
      })
      .then((chunk) => {
        chunks.push(chunk);
        if (start < file.size) {
          return readFileChunk(file, start, chunkSize, chunks);
        } else {
          return new Blob(chunks);
        }
      });
    };

    const readFile = (file) => {
      const chunkSize = 10485760; // 10MB
      const blobs = [];
      return readFileChunk(file, 0, chunkSize, blobs)
    };

However, this approach is memory consuming, because the final blob is assembled and stored in memory. If you’re using very large files, the application may crash. In this case, we don't need to assemble the chunks – we can just send them to a server with some additional data (each chunk's start and end in the result file).

Liked this article? Sign up to be notified of new blog posts.

You might also like

React Native vs. Ionic. The battle of giants in the cross-platform development field

Web Development Trends in 2018

Web vs. native vs. hybrid applications. Finding a compromise between price and performance

React Native and Ionic are main competitors among cross-platform app development frameworks. We overview the main differences, advantages, and disadvantages of each framework to help you make the right decision.
Web development trends are changing rapidly. Yesterday the multi-page websites were popular, today the trend for simplicity brought single-page apps back on the top. Read about other tendencies in web development for 2018.
Web and hybrid apps a bad, native apps are expensive. If you keep hearing this cliche, let us dispel the myth. You will be surprised how many world’s top companies have “bad” apps and are still adored by their users. With this article you will make the wise choice about your future app development.

React Native vs. Ionic. The battle of giants in the cross-platform development field

React Native and Ionic are main competitors among cross-platform app development frameworks. We overview the main differences, advantages, and disadvantages of each framework to help you make the right decision.

Web Development Trends in 2018

Web development trends are changing rapidly. Yesterday the multi-page websites were popular, today the trend for simplicity brought single-page apps back on the top. Read about other tendencies in web development for 2018.

Web vs. native vs. hybrid applications. Finding a compromise between price and performance

Web and hybrid apps a bad, native apps are expensive. If you keep hearing this cliche, let us dispel the myth. You will be surprised how many world’s top companies have “bad” apps and are still adored by their users. With this article you will make the wise choice about your future app development.