How to test Visual Studio Code extension — part 1

Image for post
Image for post

Table of contents

From this article you will learn how to:

  • how to write a complete Visual Studio Code extension test
  • find documentation, articles, and use cases to test VSCode extension
  • change extension preferences for each test case
  • programatically create and open files
  • grab information about rich features of VSCode (like CodeLens)
  • how to handle Thenable errors

Why I need VSCode extension tests?

Visual Studio Code some pretty decent API to write extensions. That is why i was able to write Assistant. A realtime linter to write easily your own code rules for annoying typos and mistakes.

The extension is now used by 52 developers around the world. Mostly thanks to the previous article I wrote with my ready-made Angular set of eight rules. I find this article a nice way to explain how Assistant can be used, because it has real life examples of code that Assistant can help with.

Also, a lots of people got the idea straight away, and gave suggestions what features they would like to have. Before i have started implementing these, a question arises. How to take care of Assistant quality while adding new features. Of course the answer is one — tests.

So yesterday i have spent some time setting up tests. Now 9 tests guard the Assistant, so that new changes won’t break how it works. And this is lovely and revealing.

But it was not so simple! In fact, i found documentation for Visual Studio Code (Vscode if you mind) extension testing is minimalistic. Also, there are only several articles about it over the Internet! But combining information from these sources and another one that i just have found out, allows to test almost everything in Visual Studio Code extension, and makes the work very smooth.

So, in this article you are about to find out, how to write VSCode extention tests! Good for you!

Visual Studio Code extension testing resources

First source of information about testing Visual Studio Code extensions is the documentation, and also a demo test, and also VSCode API documentation.

Additional articles that may help you during the process. Scroll down to continue your reading:

But it didn’t solve all of my problems. But i have found a way to get more information. Thanks to GitHub search! And this is mind blowing!

Anyways it was for me, when i used it for the first time…

You can actually look through all files stored on GitHub for fragments you are interested in! You can even provide a name of a file you look for!

All you need to do is to put “extension.test.ts” file name (since this is default, hello world test file of an extension) and class name, or anything you are looking for. And voila! A list of use-cases you can learn from.

I strongly recommend this method because it helped me solve two problems i could not solve in any other way!

Since Assistant is open source, you can just scoop through files and learn how to test extension.

Now, when you know some helpful resources, lets move on to some actual testing. VSCode provides a nice way to pass configuration to extensions. I will tell you how to change configuration for each test.

How to change the configuration for a test?

Assistant relies on settings provided either via Workspace settings file, or User settings file. Both are JSON by the way.

The problem with Workspace settings is that you need to create Workspace from test before you can apply settings to it. It makes sense. But i didn’t find a way to create a Workspace from tests. There is a question on StackOverflow about it, maybe some day it will be solved.

But what i was able to achive is to change user settings from a test. It’s not perfect, but it works. So in order to change user settings i wrote a helper function:

export async function changeConfig(settings: any) {
const assistant = vscode.workspace.getConfiguration();
await assistant.update(
"assistant",
settings,
ConfigurationTarget.Global
);
}

First think you may have noticed is that it uses async/await. It is because configuration update is asynchronous (Thenable to be precise). A lots of VSCode API calls are asynchronous, so be aware of it. Otherwise your code will run into some race condition situations.

So what does the code do? The function grabs settings you provide to it (JSON) and updates assistant extension settings with these. As you can see, configuration target is global.

If you know how to create a Workspace from a test, you can change configuration target to Workspace.

Hint!

Here i want to share with you a hint that may save your day. If you won’t create a Workspace and try to apply settings to it, there won’t be any error. That is because async/await in the latter example does not handle Thenable Promise failure.

If you want to handle failure, you need to use then/onFulfilled/onRejected flow. You can read more about it here. The error will be available in onRejected handler.

Now, when we have a way to change the configuration we can set it up. Here is an example:

await helper.changeConfig(
{
rules: [
{
"regex": "uniqueTextTest",
"message": "Lens shows up"
}
]
}
);

As you can see i set Assistant configuration, by providing an array of rules. The array has one rule. It tells that if in the visible Visual Studio Code file uniqueTextTest appears, it should show message “Lens shows up” above. Why Lens? Because the feature of Visual Studio Code that provides this functionality is called CodeLens. You can see it here, for other Assistant rule (“Define properly value…”):

Image for post
Image for post

As you can see providing configuration for each test is not so hard when you know it. What I am also using now, is a helper method to clear user settings at the end, so i am not stuck with test case settings. It is not neat, but it works for me for now. I call it from a last test in a test suite for the time being, but teardown function is also a good place to do it:

export async function clearConfig() {
await changeConfig(
{
rules: []
}
);
}

Assistant displays CodeLens for open editor files. So in order to test it, a new file needs to be opened in the test Visual Studio Code instance. To accomplish it, you can use this helper function:

export async function openFile(content: string): Promise<vscode.TextDocument> {
const document = doc(content);
vscode.window.showTextDocument(await document);
return document;
}

As you can see, it is also asynchronous. It creates a new document with a provided content, and opens it in the application.

Now, the last part is to check if a CodeLens is visible. I had some trouble finding out how to do it. It occurs you need to use built-in commands. They give the ability to fetch informations about all rich VSCode features.

For example this handy helper function fetches visible CodeLenses for a given document:

export async function getLens(uri: vscode.Uri) {
return await vscode.commands.executeCommand(
'vscode.executeCodeLensProvider',
uri) as vscode.CodeLens[];
}

How to retrieve document uri? I get it from the previously mentioned openFile helper method. Combined it looks like this:

const document = helper.openFile('seal\ndog\ncat');
const lens = await helper.getLens((await document).uri);
assert.equal(lens.length, 2);
assert.equal(lens[0].command?.title, "Dog shows up");
assert.equal(lens[1].command?.title, "Cat shows up");

So i open a file, grab lenses, and check if there are two of them, and that text they display. It looks nice, isn’t it?

!Hint
You need to run tests from Run panel (CTRL+SHIFT+D), instead of using console npm run test command because of Visual Studio Code limitations

After everything what was written, writing a test is straight away task. Here is an example of a simple test:

test('Basic lens shows up', async () => {
await helper.changeConfig(
{
rules: [
{
"regex": "uniqueTextTest",
"message": "Lens shows up"
}
]
}
);
const document = helper.openFile('uniqueTextTest\nSecond line');
const lens = await helper.getLens((await document).uri);
assert.equal(lens.length, 1);
assert.equal(lens[0].command?.title, "Lens shows up");
});

What happens here is:

  1. A rule is added to show “Lens shows up” when “uniqueTextTest” occurs in the document
  2. A document with “uniqueTextTest” in a first line, and “Second line” in the second line is opened
  3. Test checks if precisely one CodeLens shows up with a text “Lens shows up” as expected

With all of this knowledge writing tests was very easy. And now Assistant is fully covered:

Image for post
Image for post

This article is a good starting point to test Visual Studio Code extensions. You know how to change settings, open files and read information about rich VSCode features.

While researching information was a little bit tedious due to lack of online resources, GitHub repository search saved the day. Writing tests is very easy when you have all the information from this article.

If you will write an extension please let me know!
Don’t forget also to clap for the article and test Assistant
out!
New features coming soon!

Written by

Senior software development consultant. Programming for 20 years. TOP 2% of StackOverflow users. 2 million views on Quora. Currently Angular, TypeScript etc

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store