We’ll be creating a tiny extension which will run in any of the major web browsers currently supporting Web Extensions: Chrome, Firefox and Opera. We will demonstrate how this extension can be installed in all major browsers. Also, we’ll show basic tips on how to maintain a distinct base in all of them. How to troubleshoot in each browser.
Update 08/12/2016: To answer some comments I got on Twitter, I don’t cover Safari in this article as it doesn’t support the same extension model as others: https://developer.apple.com/reference/safariextensions. To add Vivaldi to the article, I updated it.
Because there are already many great resources provided by each vendor, I won’t go into detail about extensions.
– Google: https://developer.chrome.com/extensions- Microsoft: https://developer.microsoft.com/en-us/microsoft-edge/platform/documentation/extensions/ and I advise also this great overview video: Building Extensions for Microsoft Edge
– Mozilla: https://developer.mozilla.org/en-US/Add-ons/WebExtensions and https://wiki.mozilla.org/WebExtensions
– Opera: https://dev.opera.com/extensions/getting-started/
– Brave: https://github.com/brave/browser-laptop/wiki/Developer-Notes-on-Installing-or-Updating-Extensions
These resources will help you if your first extension was not built or if it’s new to you. You don’t have to worry about how difficult it is to construct them.
Let’s make a POC of an extension by using AI/Computer Vision. It will assist the blind in analyzing web pages images.
With just a few lines code you can build some amazing features for the browser. For me, accessibility is a major concern. In fact I spent time researching how to make a breakout game available using Web Audio & SVG.
I was still looking for something that would aid blind people in an even more general manner. While I was listening to Chris Heilmann speak in Lisbon about pixels and hidden meanings in pixels, it inspired me.
With today’s Artificial Intelligence (AI) , as well as the text-to-speech technologies more exposed in browsers with Web Speech API, or also using remote cloud services, it is possible to quickly build a solution for an image that has an incorrectly or missing ALT text.
My proof of concept will just extract all images from one web page (the active tab), and show the thumbnails as a list. It will ask the Computer Vision API for text description of each image. The Web Speech API and Bing Speech API will then be used to share the information with listeners.
The video below shows how it works in Edge, Chrome Firefox, Opera, and Brave! Please enable sound to see how it works
It’ll be obvious that even though the Computer Vision API analyzes some CGI images, its accuracy is remarkable. This industry has really made great progress over the past few months.
For me, these services have been used:
– Microsoft Cognitive Services offers the Computer Vision API for no cost (with a limited number of users). For this to work, generate a new key and then replace the TODO line with the generated key. To have an idea of what this API could do, play with it: https://www.captionbot.ai
You can find the code of this small browser’s extension on my github: http://github.com/davrous/dareangel
If you want to try other services, or modify your code for them to work with it, feel free.
Tip: To make sure your code is compatible with all browsers
The namespace is used for extensions APIs like Chrome.tabs.
As I mentioned, however, the Extension API model currently is being standardized to , and certain browsers such as Edge then define the MsBrowser namespacing.
You will also have to use the API subset that all browsers support. You can use this example:
– Microsoft Edge supports the current list: https://developer.microsoft.com/en-us/microsoft-edge/platform/documentation/extensions/api-support/
– Mozilla Firefox shares its current Chrome incompatibilities: https://developer.mozilla.org/en-US/Add-ons/WebExtensions
Let’s look at this extension’s architecture. It should be helpful to you if this is your first time using browser extensions.
This JSON defines the minimum necessary to allow extension loading in all browsers. If you want to load an extension in Edge then it is necessary that you specify an author property. It is also necessary to follow the same structure when creating icons.
Below are some resources that will assist you in creating a file called Manifest compatible with all browsers.
Simple Content script:
The console logs into first to verify the extension is properly loaded via F12.
After waiting for a message, the UI page sends a request to requestImages . It returns a list with their URL if the images are greater than 64×64.
This popup is simple. It will display the images from the content script in a container. It loads the start.js script which immediately creates an instance of dareangel.dashboard.js to send a message to the content script to get the URLs of the images of the current tab displayed.
The UI page requests URLs to the script. Here is the code:
Each image we create will trigger an event, if the focus is on the Computer Vision API.
It is as easy as dialing XHR
You’ll find out the workings of this Computer Vision API here:
Analyzing an image Version 1.0 will tell you everything you can do using this technology
Computer Vision API (v1.0) This will provide an interactive interface on a website that shows you how to call REST API. It includes the JSON properties as well as the JSON objects you’ll receive in return. Understanding the basics of how this works and what you’ll do with it is helpful.
The description featureof our API is used in this case. Based on your selections, you can see that the callback also indicates that we will attempt to use the HTML Speech API and the Text-To Speech services.
The global workflow is now complete.
The extension can be loaded in every browser
Let’s quickly review how to install the extension in each browser.
Download or clone this small extension somewhere on your hard drive from my github: https://github.com/davrous/dareangel
Please also modify dareangel.dashboard.js to add at least a Computer Vision API key. If you do not, then the extension cannot display any images from your web page.
For extensions to work in Edge you will need at minimum a Windows 10 Anniversary Update. You can then:
Type into the address bar. You can check the Enable Extension Developer Features.
To clone your repo, click on “…”. Here’s what you will see:
– Right click the newly loaded extension to enable the ” Display button near the address bar“.
The Reload extension buttons are useful for extension development. You don’t have to delete/reinstall your extension while you are developing it. Simply click reload and refresh the extension.
– Navigate to http://www.babylonjs.com and click on the Dare Angel (DA) button to do the same demo as shown in the video.
– Open Chrome and navigate to “chrome://extensions” and enable the “Developer mode”
You can click on ” Load Unpacked Extension“, and select the folder from which your extension was extracted.
– Navigate to http://www.babylonjs.com and open the extension to check that it works fine.
Two options are available. You have two options. The first one is to temporarily load the extension. This process works just like in Edge and Chrome.
– Launch Firefox. Navigate to ” About:debugging“, and then click on ” Temporary Addon“.
– Go to the Extension folder and choose the manifest.json files.
This is it. Navigate to http://www.babylonjs.com to test the extension.
However, this method has the drawback that you have to restart your extension every time you close your browser. The XPI packaging is another option.
– Start Opera, navigate to About://extensions, and click the ” developer mode“.
– Select ” Load Unpacked Extension…” to choose the folder in which you have extracted your extension
– Navigate once again to http://www.babylonjs.com and open the extension to test it.
– Open Vivaldi and navigate to “vivaldi://extensions” and enable the “Developer mode”
– Select ” Load Unpacked Extension…” to choose the folder in which you have extracted your extension
– Go to http://www.babylonjs.com and open the extension to test it.
Brave does not have an embedded “developer mode”, which allows you to upload unsigned extensions. You can get the public version on their website. You must make your own copy by following the steps.
– Clone this repo: https://github.com/brave/browser-laptop in ~/projects on your local hard drive
To install , open the Browser-laptop folder. Please be patient as it will download many dependencies.
Navigate into the folder ” application“, and then open the extensions.js file with your text editor. You will need to locate the lines where the extension registration code is. For my situation, the last two lines have been added:
Copy the extension to the ” application/extensions folder.
– Go to the ” 2 command prompts folder. Start ” .npm.run“, and wait until webpack finishes building Brave electron. The message should read ” bundle is now valid“. It should say “strong>webpack: bundle is now VALID/strong>”. If it does not, there are some serious problems.
Next, open ” start” in the second window. This will launch Brave’s slightly customized version.
Brave: Navigate to about:extensions. You should then see the extension loaded and displayed in the address line.
Debugging the extensions in every browser
To debug the client-script part of the page’s context, all you need is to open F12. Click on ” Debugger Tab and locate the extension folder.
To debug the code, open the dareangel.client.js script file. Set up breakpoints as necessary.
For debugging extensions that are creating separate tabs to perform their job, press F12 in this tab.
You will need your extension ID to analyze the popup page. To do this, go to Extension property and look for the ID property.
Then, you need to type in the address bar something like: ms-browser-extension://ID_of_your_extension/yourpage.html . In our case, it would be: ms-browser-extension://DareAngel_vdbyzyarbfgh8/dashboard.html for instance. Simply use F12 to access this page.
Opera and Chrome both rely upon the same Blink base code, so they have the same debugging process. Even though Brave & Vivaldi might be different forks, most often they have the same debug procedure.
Open HTML12 to open the Client Script Part (or CTRL+SHIFT+I within Opera)
Click on the ” content scripts tab to locate your extension directory. Debug the code you wish to open in the script editor.
To Debug a Tab Your Extension Would Create, it is the exact same as Edge. Simply use HTML12.
Opera and Chrome allow you to inspect the popup pages. Right-click the extension button near the address bar, choose “Inspect Popup” and then open the HTML panel. Click inside the popup to go to “Inspect.” Vivaldi supports right-click -> inspect within the HTML panel once it is opened.
The process is the same for Brave. To find your extension’s GUID in ” >extensions“, first search:
And then open in a separated tab the page you’d like to debug like in my case: “chrome-extension://bodaahkboijjjodkbmmddgjldpifcjap/dashboard.html” and use F12.
To help with the layout you can use the ” SHIFT+F8“, which will allow you to inspect all of Brave’s frames. Brave, an electron app with React.js will be revealed! And you’ll discover that Brave uses React.js to create an electron app!
For instance, notice the “datareactroot” attribute.
Note – I had to modify the CSS extension for Brave because it displays popups with transparent backgrounds and my images were too tall. Brave’s case, I have reduced it to four elements.
Mozilla has a really great documentation on Web Extensions Debugging: https://developer.mozilla.org/fr/Add-ons/WebExtensions/Debugging
For the client script part, it’s the same story as in Edge/Chrome/Opera/Brave, simply open F12 on the tab you’d like to debug and you’ll find a “moz-extension://guid” section with your code to debug:
If you are looking to Debugge a Tab, an extension could create (similar to the Vorlon.js page Analyzer extension), just use .
The popup can be debugged using documentation. It is more involved but clearly explained in the Debugging Popups section.
It’s amazing to realize that we can now use our regular JS/CSS/HTML skill set, and build fantastic extensions . We also have the same base for all browsers .
As much as I can, I have tried to make this proof-of-concept as accessible as I possibly could from the vorlon.js extension.
You can ping me on Tweet at @davrous to give feedback.