If you use GitHub’s official copilot.vim extension, you should know that disabling the plugin doesn’t prevent initial telemetry calls to their cloud every time you open up vim. There are a couple of privacy and anonymity implications to this, and fortunately, the patch is pretty simple.
Skip to the fix.
Contents
The Proof
You can run the following command to view domains being queried in DNS requests made from your system:
$ tshark -i ${interface} -f "udp port 53 or tcp port 53" -T fields -e dns.qry.name
# Replace ${interface} with your actual network interface, of course.
Opening up vim while tshark runs will reveal that there are numerous calls to GitHub’s telemetry servers. In particular, the following domains are contacted:
api.github.com
copilot-telemetry.githubusercontent.com
telemetry.individual.githubcopilot.com
proxy.individual.githubcopilot.com
api.individual.githubcopilot.com
dc.services.visualstudio.com
telemetry.individual.githubcopilot.com
You can additionally confirm that Copilot is disabled by running the following inside vim:
The Fix
Add the following lines to the top of plugin/copilot.vim in the plugin’s directory:
if !get(g:, 'copilot_enabled', 1)
finish
endif
Alternatively, if you’d like something you can copy-paste into your terminal directly:
cd $(find ~/.vim/ -name "copilot.vim" -type d) && cat <<EOF | git apply -
diff --git a/plugin/copilot.vim b/plugin/copilot.vim
index f29f800..b28c3b2 100644
--- a/plugin/copilot.vim
+++ b/plugin/copilot.vim
@@ -1,3 +1,7 @@
+if !get(g:, 'copilot_enabled', 1)
+ finish
+endif
+
if exists('g:loaded_copilot')
finish
endif
EOF
This short-circuits the plugin before any additional code is run, properly preventing the telemetry calls upon startup when the plugin is disabled:
Afterwards, you can start tshark and open up vim again, and observe no telemetry traffic:
The copilot.vim repository doesn’t accept pull requests, so I unfortunately have no way of getting this patch into the codebase.
Technical Analysis
Thank you to my friends from Iris for helping me with their reverse engineering knowledge and sanity checks. Specifically, thank you to coded, nope, and sera.
Running a quick grep for githubcopilot.com finds results in copilot-language-server/dist/main.js:
$ grep -oiR "githubcopilot.com" . | sort | uniq
./copilot-language-server/dist/main.js:githubcopilot.com
Trying to analyze the file itself is challenging; it’s minified and obfuscated, and the source code is not available. This is the GitHub Copilot Language Server and is necessary for the GitHub Copilot plugin to function.
After working backwards from here, it was discovered that the load order goes:
plugin/copilot.vim
autoload/copilot.vim
autoload/copilot/client.vim
copilot-language-server/dist/language-server.js
copilot-language-server/dist/main.js
plugin/copilot.vim registers an autocmd on VimEnter calling copilot#Init():
augroup github_copilot
autocmd!
autocmd FileType * call s:Event('FileType')
autocmd InsertLeavePre * call s:Event('InsertLeavePre')
autocmd BufLeave * if mode() =~# '^[iR]'|call s:Event('InsertLeavePre')|endif
autocmd InsertEnter * call s:Event('InsertEnter')
autocmd BufEnter * if mode() =~# '^[iR]'|call s:Event('InsertEnter')|endif
autocmd BufEnter * call s:Event('BufEnter')
autocmd CursorMovedI * call s:Event('CursorMovedI')
autocmd CompleteChanged * call s:Event('CompleteChanged')
autocmd ColorScheme,VimEnter * call s:ColorScheme()
autocmd VimEnter * call s:MapTab() | call copilot#Init()
autocmd BufUnload * call s:Event('BufUnload')
autocmd VimLeavePre * call s:Event('VimLeavePre')
autocmd BufReadCmd copilot://* setlocal buftype=nofile bufhidden=wipe nobuflisted nomodifiable
autocmd BufReadCmd copilot:///log call copilot#logger#BufReadCmd() | setfiletype copilotlog
augroup END
autoload/copilot.vim defines copilot#Init(), which calls the local s:Start(), which calls copilot#client#New():
function! copilot#Init(...) abort
call copilot#util#Defer({ -> exists('s:client') || s:Start() })
endfunction
...
function! s:Start() abort
if s:Running() || exists('s:client.startup_error')
return
endif
let s:client = copilot#client#New()
endfunction
autoload/copilot/client.vim defines copilot#client#New(), which constructs a command from the local s:Command() and then calls it using job_start (docs) to hand off execution to the JS:
" This function primarily revolves around grabbing Node and the JS path.
let s:script_name = 'copilot-language-server/dist/language-server.js'
let s:pkg_name = '@github/copilot-language-server'
function! s:Command() abort
...
return [node, script, '']
endfunction
...
" This function hands off execution to the JS.
function! copilot#client#New() abort
...
let [node, argv, command_error] = s:Command()
...
let instance.job = job_start(command, {
\ 'cwd': copilot#job#Cwd(),
\ 'noblock': 1,
\ 'stoponexit': '',
\ 'in_mode': 'lsp',
\ 'out_mode': 'lsp',
\ 'out_cb': { j, d -> copilot#util#Defer(function('s:OnMessage'), instance, d) },
\ 'err_cb': function('s:OnErr', [instance]),
\ 'exit_cb': { j, d -> copilot#util#Defer(function('s:OnExit'), instance, d) },
\ })
...
endfunction
Finally, copilot-language-server/dist/language-server.js loads copilot-language-server/dist/main.js:
function main() {
const argv = process.argv.slice(2);
const version = process.versions.node;
const [major, minor] = version.split('.').map(v => parseInt(v, 10));
if (major > minMajor || (major === minMajor && minor >= minMinor)) {
return require('./main').main();
}
...
}
main();
It is in copilot-language-server/dist/main.js that these telemetry domains are referenced, though the functions and payloads thereof are difficult to analyze since the Copilot Language Server is minified and obfuscated, and source code is not available. The domains can be extracted with grep:
$ grep -Eo "[[:alnum:]][[:alnum:].-]*\.com\"" main.js | tr -d '"' | sort | uniq
api-model-lab.githubcopilot.com
api.anthropic.com
api.github.com
api.githubcopilot.com
azure.com
ciamlogin.com
cloudforge.com
copilot-proxy.githubusercontent.com
copilot-telemetry.githubusercontent.com
copilot.github.com
dc-int.services.visualstudio.com
dc.services.visualstudio.com
dev.azure.com
ghe.com
git.cloudforge.com
github.com
in.applicationinsights.azure.com
inference.ml.azure.com
login-us.microsoftonline.com
login.microsoft.com
login.microsoftonline.com
models.ai.azure.com
mongodb.com
onmicrosoft.com
openai.azure.com
origin-tracker.githubusercontent.com
rt.services.visualstudio.com
services.visualstudio.com
ssh.dev.azure.com
uploads.github.com
visualstudio.com
vs-ssh.visualstudio.com
westeurope-5.in.applicationinsights.azure.com
westus-0.in.applicationinsights.azure.com
It looks like the initial telemetry calls are executed regardless of whether or not the plugin is enabled since the JS is loaded unconditionally. In this way, GitHub has chosen to interpret “disabling Copilot” as in “disabling Copilot suggestions,” as opposed to “disabling the Copilot plugin.”
The fix works by short-circuiting the plugin with a quit condition before any of the plugin’s code is allowed a chance to execute. By not loading the code, we are actually disabling the Copilot plugin.
Protocol Analysis
I was curious, so I analyzed the telemetry call payloads to better understand what data is being phoned home to the cloud. I set up a proxy with mitmproxy and set some environment variables used by vim to proxy the traffic, allowing me to inspect SSL-encrypted packets by swapping the certificates:
$ sudo trust anchor --store mitmproxy-ca-cert.pem # Get the cert from mitmproxy.
$ sudo trust anchor --remove mitmproxy-ca-cert.pem # For cleanup afterwards.
$ export HTTP_PROXY="http://${IP}:8080"
$ export HTTPS_PROXY="http://${IP}:8080"
$ export http_proxy="$HTTP_PROXY"
$ export https_proxy="$HTTPS_PROXY"
$ vim
Note that the initial call to registry.npmjs.org will fail; I suspect the certificate is pinned. If you’re following along, add it to ignore_hosts in mitmproxy, or else the plugin will enter a failed state before you can capture the telemetry packets.
After setting up the proxy and starting up vim without the patch (in order to analyze the initial telemetry calls), the traffic and their decrypted payloads can be analyzed:
A few observations:
- The client presents a bearer token, and the API replies with a list of endpoints and a tracking ID (among other information). Importantly, the API has taken note of the client’s IP address and even presents this in the
"token" field of the response.
- The client gets a list of embedding models, and the API replies. During my analysis, I only got
metis-1024-I16-Binary.
- The client gets information about their account, and the API replies with their username, tracking ID, quota status, and other information.
- The client sends off some information about their system, including their application versions, OS/kernel versions, CPU, architecture, etc. There’s a field called
"client_machineid" which appears to be a unique identifier for the specific client device.
- The client gets a list of coding models, and the API replies.
- The client gets a list of Copilot configurations to collect, and the API replies.
- The client sends off the list of Copilot configurations specified by the API.
- The client gets more detailed information about coding models and token prices, and the API replies.
- The client again sends off some information about their system and Copilot configurations.
This looks pretty standard for telemetry.
Assessment
This traffic is pretty standard for telemetry. The concerning part is that this exchange occurs regardless of whether or not the plugin is enabled; this presents some concerning privacy and anonymity implications.
A user who has “disabled” the Copilot plugin will still be making telemetry calls every time they open up vim, and GitHub will be able to identify them to their username and IP address. If a user is using vim behind a proxy such as a VPN or TOR, GitHub will be able to identify them regardless. This can be used to de-anonymize users.
Anyways, GitHub doesn’t accept PRs.
Happy hacking.