Test e2e with vue3 and cypress is flaky

GitLab CI/CD
,
1

Hello all,

I try to run my e2e test with cypress but my test is flacky.
First of all cypress without browser needs to have an application that running like localhost:4173 so you need to build your app and run your app and then you can run your test.

I try a lot of way to resolve my flacky but without result…
I put you my jobs here.

- The easiest way

test-e2e:
  stage: test
  image: cypress/browsers:node-20.9.0-chrome-118.0.5993.88-1-ff-118.0.2-edge-118.0.2088.46-1
  script:
    - npm clean-install
    - npm run preview & || npm run preview > /dev/null & 
    - sleep 5
    - npx cypress run --browser firefox

And I got this error

$ npm run preview > /dev/null &
$ npx cypress run --browser firefox
[STARTED] Task without title.
[SUCCESS] Task without title.
DevTools listening on ws://127.0.0.1:32961/devtools/browser/70110253-1c02-45d1-ac4d-b1077b55bb35
Missing baseUrl in compilerOptions. tsconfig-paths will be skipped
Cypress could not verify that this server is running:
  > http://localhost:4173
We are verifying this server because it has been configured as your baseUrl.
Cypress automatically waits until your server is accessible before running tests.
We will try connecting to it 3 more times...
We will try connecting to it 2 more times...
We will try connecting to it 1 more time...

Finaly I found a bug I vue : Vite server running in a background process stopped for consuming shell's input stream · Issue #15287 · vitejs/vite · GitHub, si I take the decision to run my app with other way

- With npx serve

test-e2e:
  stage: test
  image: cypress/browsers:node-20.9.0-chrome-118.0.5993.88-1-ff-118.0.2-edge-118.0.2088.46-1
  script:
    - npm clean-install
    - npx serve dist -p 4173 &
    - npx cypress run --browser firefox

And I got this error

$ npx serve dist -p 4173 &
$ sleep 5
 INFO  Accepting connections at http://localhost:4173
$ npx cypress run --browser firefox
[STARTED] Task without title.
[SUCCESS] Task without title.
DevTools listening on ws://127.0.0.1:43059/devtools/browser/9de40ff2-8459-4504-82cb-daf786ea222f
Missing baseUrl in compilerOptions. tsconfig-paths will be skipped
Cypress could not verify that this server is running:
  > http://localhost:4173
We are verifying this server because it has been configured as your baseUrl.
Cypress automatically waits until your server is accessible before running tests.
We will try connecting to it 3 more times...
We will try connecting to it 2 more times...
We will try connecting to it 1 more time...
Cypress failed to verify that your server is running.
Please start this server and then run Cypress again.

- With pm2

test-e2e:
  stage: test
  image: cypress/browsers:node-20.9.0-chrome-118.0.5993.88-1-ff-118.0.2-edge-118.0.2088.46-1
  script:
    - npm install -g pm2
    - npm clean-install
    - pm2 start ecosystem.config.js
    - sleep 5
    - npx cypress run --browser firefox

And I got this error

[PM2] Spawning PM2 daemon with pm2_home=/root/.pm2
[PM2] PM2 Successfully daemonized
[PM2][WARN] Applications vite-app not running, starting...
[PM2] App [vite-app] launched (1 instances)
┌────┬─────────────┬─────────────┬─────────┬─────────┬──────────┬────────┬──────┬───────────┬──────────┬──────────┬──────────┬──────────┐
│ id │ name        │ namespace   │ version │ mode    │ pid      │ uptime │ ↺    │ status    │ cpu      │ mem      │ user     │ watching │
├────┼─────────────┼─────────────┼─────────┼─────────┼──────────┼────────┼──────┼───────────┼──────────┼──────────┼──────────┼──────────┤
│ 0  │ vite-app    │ default     │ 5.3.0   │ fork    │ 97       │ 0s     │ 0    │ online    │ 0%       │ 19.1mb   │ root     │ disabled │
└────┴─────────────┴─────────────┴─────────┴─────────┴──────────┴────────┴──────┴───────────┴──────────┴──────────┴──────────┴──────────┘
$ sleep 10
$ npx cypress run --browser firefox
[STARTED] Task without title.
[SUCCESS] Task without title.
DevTools listening on ws://127.0.0.1:42041/devtools/browser/7eef3e21-7065-433c-8bcb-2e61901252ea
Missing baseUrl in compilerOptions. tsconfig-paths will be skipped
====================================================================================================
  (Run Starting)
  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Cypress:        13.3.0                                                                         │
  │ Browser:        Firefox 118 (headless)                                                         │
  │ Node Version:   v20.9.0 (/usr/local/bin/node)                                                  │
  │ Specs:          1 found (LoginAndFetchProjectsScenario.cy.ts)                                  │
  │ Searched:       cypress/e2e/**/*.{cy,spec}.{js,jsx,ts,tsx}                                     │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘
────────────────────────────────────────────────────────────────────────────────────────────────────
                                                                                                    
  Running:  LoginAndFetchProjectsScenario.cy.ts                                             (1 of 1)
  LoginPage
    1) when user success to login he is redirected to the home page
  0 passing (6s)
  1 failing
  1) LoginPage
       when user success to login he is redirected to the home page:
     AssertionError: Timed out retrying after 4000ms: Expected to find element: `h1`, but never found it.

In hope of a solution,

Best regards,

Théo.