Selenium IDE exportに「not worry about vendor lock-in when using mabl.」とあるように、mablは基本的にいろんなものをダウンロード/エクスポートできます。それぞれどういうものかをみてみましょう。基本CSV形式です。
Planのダウンロード
"APPLICATION_ID","APPLICATION_NAME","DEPLOYMENT_ID","PLAN_ID","PLAN_NAME","PLAN_RUN_ID","PLAN_URL","RUNTIME_MILLIS","STARTING_URL","START_TIME","STATUS","STOP_TIME","SUCCESS","TAGS","TRIGGER_TYPE","TRIGGER_USER_EMAIL","TRIGGER_USER_ID","WORKSPACE_ID","WORKSPACE_NAME"
"XXXX","mabl sandbox","YYYY","ZZZZ","mabl sandbox plan","AAAAA","https://app.mabl.com/FFFFFF/test/plans/AAAA","81306","https://sandbox.mabl.com/","1583035204303","succeeded","1583035285609","false","","mabl_scheduler","","v1_8","GGGGG","daipresents workspace"
"XXXX","mabl sandbox","YYYY","ZZZZ","mabl sandbox plan","BBBBB","https://app.mabl.com/FFFFFF/test/plans/AAAA","74473","https://sandbox.mabl.com/","1582948764871","succeeded","1582948839344","false","","mabl_scheduler","daipresents","GGGGG","EEEE","daipresents workspace"Plan一覧のダウンロード
CLIを利用。
% mabl plans list --limit 1
┌──────────────────────────┬─────────────────────────────────┬─────────────────────────┐
│ ID │ Name │ Created time │
├──────────────────────────┼─────────────────────────────────┼─────────────────────────┤
│ Ccix0Meyd8znbQ2PK80Q6w-p │ Verify home page load and login │ Aug 11th 2021, 07:12:01 │
└──────────────────────────┴─────────────────────────────────┴─────────────────────────┘
1 plans returnedテストケース一覧のダウンロード
"id","journey","percentagePassed","planCount","createdBy","createdDate","description"
"XXXXXXXXXXXXXXXXXXX","Custom HTTP Header の確認","100","1","Dai Fujihara","Feb 10, 2020","Custom HTTP Headerの確認。JSからのアクセス（navigator.userAgent）と HTTPヘッダの中身を確認しています。
参考：https://atomic-temporary-22783153.wpcomstaging.com/2020/creating-mobile-web-tests/"
"YYYYYYYYYYYYYYYYYYY","API request and response validation / APIのテスト","100","1","Dai Fujihara","Jan 30, 2020","APIを叩いてそのレスポンスをチェックしています。これによってテストデータをAPI経由で準備したり、データの状態を変更してテストを進められるためテストが簡潔になります。
参考： https://atomic-temporary-22783153.wpcomstaging.com/2020/mabl-api-requests-response-validation/"
"ZZZZZZZZZZZZZZZZZZZZZ","Mobile View - モバイルページの確認","100","1","Dai Fujihara","Jan 29, 2020","レスポンシブルページをモバイルブラウザで表示します。mablではCustom HTTP HeaderやViewportの設定が可能です。
参考：https://atomic-temporary-22783153.wpcomstaging.com/2020/creating-mobile-web-tests/
"
"DEFAULTTTTTTTTTTTTTTT","Visit all linked pages within the app","70","3","Dai Fujihara","Jan 7, 2020","Starting at the primary url, crawl all links within the application's domain, validate that the linked pages load."
"DEFAULTTTTTTTTTTTTTTT","Visit home page","100","3","Dai Fujihara","Jan 7, 2020","Attempt to load the primary url in a browser, validate that it loads, verify that login works (if provided)."journeyはTestを指している。昔、mablはTestをJourneyと呼んでいたそう。
テストケースのダウンロード（CSV)
,Step Number, Step Type, Description, Annotation, Note
0, Echo, Echo: "# ログイン（成功）の観点をチェック", -, -
1, Echo, Echo: "## 共通処理をここで定義", -, -
2, VisitUrl, Visit URL "{{@app.url}}", -, "{{@web.defaults.url}}"を開く
3, SetViewport, Set viewport size to width 1600, -, 画面の幅を1600pxに設定
4, Echo, Echo: "## ログイン処理", -, -
5, Click, Click on the <span> element with text "SIMULATED LOGIN", -, "SIMULATED LOGIN"をクリックする
6, WaitUntil, Wait up to 15 seconds until target element matching Text attribute is present, -, ログインボタンが表示されることまでまつ
7, EnterText, Insert value of variable "flow.email" into the "email" text field, -, Emailを入力
8, EnterText, Insert value of variable "flow.password" into the "password" text field, -, パスワードを入力
9, Click, Click on the "LOG IN" button, -, ログインボタンをクリック
10, WaitUntil, Wait up to 15 seconds until target element matching Text attribute is present, -, 次の画面が表示されるまで待つ
11, Echo, Echo: "## ログイン成功時の確認", -, -
12, AssertPresent, Assert presence of the <span> element with text "LOG OUT", -, "LOG OUT"ボタンが表示されていることを確認テストケースのダウンロード（JSON)
{
"id": "jXCALY2kE0BldpXSbDaCkQ-j:10",
"name": "mabl > ログイン",
"description": "",
"steps": [
{
"Echo": {
"description": "Echo: \"# ログイン（成功）の観点をチェック\""
}
},
{
"Echo": {
"description": "Echo: \"## 共通処理をここで定義\""
}
},
{
"VisitUrl": {
"description": "Visit URL \"{{@app.url}}\"",
"annotation": {
"description": "\"{{@web.defaults.url}}\"を開く"
}
}
},
{
"SetViewport": {
"description": "Set viewport size to width 1600",
"annotation": {
"description": "画面の幅を1600pxに設定"
}
}
},
{
"Echo": {
"description": "Echo: \"## ログイン処理\""
}
},
{
"Click": {
"description": "Click on the <span> element with text \"SIMULATED LOGIN\"",
"annotation": {
"description": "\"SIMULATED LOGIN\"をクリックする"
}
}
},
{
"WaitUntil": {
"description": "Wait up to 15 seconds until target element matching Text attribute is present",
"annotation": {
"description": "ログインボタンが表示されることまでまつ"
}
}
},
{
"EnterText": {
"description": "Insert value of variable \"flow.email\" into the \"email\" text field",
"annotation": {
"description": "Emailを入力"
}
}
},
{
"EnterText": {
"description": "Insert value of variable \"flow.password\" into the \"password\" text field",
"annotation": {
"description": "パスワードを入力"
}
}
},
{
"Click": {
"description": "Click on the \"LOG IN\" button",
"annotation": {
"description": "ログインボタンをクリック"
}
}
},
{
"WaitUntil": {
"description": "Wait up to 15 seconds until target element matching Text attribute is present",
"annotation": {
"description": "次の画面が表示されるまで待つ"
}
}
},
{
"Echo": {
"description": "Echo: \"## ログイン成功時の確認\""
}
},
{
"AssertPresent": {
"description": "Assert presence of the <span> element with text \"LOG OUT\"",
"annotation": {
"description": "\"LOG OUT\"ボタンが表示されていることを確認"
}
}
}
]
}テストケースのダウンロード（YAML)
id: 'jXCALY2kE0BldpXSbDaCkQ-j:10'
name: mabl > ログイン
description: ''
steps:
- Echo:
description: 'Echo: "# ログイン（成功）の観点をチェック"'
- Echo:
description: 'Echo: "## 共通処理をここで定義"'
- VisitUrl:
description: 'Visit URL "{{@app.url}}"'
annotation:
description: '"{{@web.defaults.url}}"を開く'
- SetViewport:
description: Set viewport size to width 1600
annotation:
description: 画面の幅を1600pxに設定
- Echo:
description: 'Echo: "## ログイン処理"'
- Click:
description: Click on the <span> element with text "SIMULATED LOGIN"
annotation:
description: '"SIMULATED LOGIN"をクリックする'
- WaitUntil:
description: >-
Wait up to 15 seconds until target element matching Text attribute is
present
annotation:
description: ログインボタンが表示されることまでまつ
- EnterText:
description: Insert value of variable "flow.email" into the "email" text field
annotation:
description: Emailを入力
- EnterText:
description: Insert value of variable "flow.password" into the "password" text field
annotation:
description: パスワードを入力
- Click:
description: Click on the "LOG IN" button
annotation:
description: ログインボタンをクリック
- WaitUntil:
description: >-
Wait up to 15 seconds until target element matching Text attribute is
present
annotation:
description: 次の画面が表示されるまで待つ
- Echo:
description: 'Echo: "## ログイン成功時の確認"'
- AssertPresent:
description: Assert presence of the <span> element with text "LOG OUT"
annotation:
description: '"LOG OUT"ボタンが表示されていることを確認'
テストケースのダウンロード（SIDE)
{
"url": "https://sandbox.mabl.com/",
"version": "2.0",
"name": "mabl > ログイン",
"tests": [
{
"id": "852afcfdae653511fe5ecdf7ae4fd8d9",
"name": "mabl > ログイン",
"commands": [
{
"id": "3accba5308ecc608c14f3beb3b71a31b",
"comment": "",
"command": "store",
"target": "https://sandbox.mabl.com/",
"targets": [],
"value": "web.defaults.url"
},
{
"id": "4e65700548b37924cad1e8c1d2a1e0f5",
"comment": "Echo: \"# ログイン（成功）の観点をチェック\"",
"command": "echo",
"target": "# ログイン（成功）の観点をチェック",
"targets": [],
"value": ""
},
{
"id": "261ff28e12d9d8acc2d012b49c79e8c3",
"comment": "Echo: \"## 共通処理をここで定義\"",
"command": "echo",
"target": "## 共通処理をここで定義",
"targets": [],
"value": ""
},
{
"id": "ebe1742ae090ff8809f52d512df4d939",
"comment": "Visit URL \"{{@app.url}}\"",
"command": "open",
"target": "${web.defaults.url}",
"targets": [],
"value": ""
},
{
"id": "22bb3687d905a457d901243959414403",
"comment": "Set viewport size to width 1600",
"command": "setWindowSize",
"target": "1600x1080",
"targets": [],
"value": ""
},
{
"id": "dc2990aa43042f3997af77fc99c5095c",
"comment": "Echo: \"## ログイン処理\"",
"command": "echo",
"target": "## ログイン処理",
"targets": [],
"value": ""
},
{
"id": "e81c1abb311267844c2f61fd4ac8c7cd",
"comment": "Click on the <span> element with text \"SIMULATED LOGIN\"",
"command": "click",
"target": "xpath=//html[1]/body[1]/div[1]/div[2]/div[1]/div[1]/div[1]/div[2]/div[1]/a[13]/span[1]",
"targets": [],
"value": ""
},
{
"id": "bd7fdc2b707135bedf1e3ec06d6ce6de",
"comment": "Wait up to 15 seconds until target element matching Text attribute is present",
"command": "assertElementPresent",
"target": "xpath=//html[1]/body[1]/div[1]/div[2]/div[1]/div[2]/form[1]/button[1]/span[1]",
"targets": [],
"value": ""
},
{
"id": "6b5e30d76f2c332d120fce217f5e6982",
"comment": "Insert value of variable \"flow.email\" into the \"email\" text field",
"command": "type",
"target": "xpath=//html[1]/body[1]/div[1]/div[2]/div[1]/div[2]/form[1]/div[1]/div[1]/input[1]",
"targets": [],
"value": "${flow.email}"
},
{
"id": "c6389c7116f612555a2e0bcfbd3be920",
"comment": "Insert value of variable \"flow.password\" into the \"password\" text field",
"command": "type",
"target": "xpath=//html[1]/body[1]/div[1]/div[2]/div[1]/div[2]/form[1]/div[2]/div[1]/input[1]",
"targets": [],
"value": "${flow.password}"
},
{
"id": "8b46450db11f30da0957aa8230696af3",
"comment": "Click on the \"LOG IN\" button",
"command": "click",
"target": "xpath=//html[1]/body[1]/div[1]/div[2]/div[1]/div[2]/form[1]/button[1]",
"targets": [],
"value": ""
},
{
"id": "bd7fdc2b707135bedf1e3ec06d6ce6de",
"comment": "Wait up to 15 seconds until target element matching Text attribute is present",
"command": "assertElementPresent",
"target": "xpath=//html[1]/body[1]/div[1]/div[2]/div[1]/div[2]/button[1]/span[1]",
"targets": [],
"value": ""
},
{
"id": "00038cde5e49ac1c9bfb609875a29b70",
"comment": "Echo: \"## ログイン成功時の確認\"",
"command": "echo",
"target": "## ログイン成功時の確認",
"targets": [],
"value": ""
},
{
"id": "d2f1194a2bc834da5a43142b3ecd9f11",
"comment": "Assert presence of the <span> element with text \"LOG OUT\"",
"command": "assertElementPresent",
"target": "xpath=//html[1]/body[1]/div[1]/div[2]/div[1]/div[2]/button[1]/span[1]",
"targets": [],
"value": ""
}
]
}
],
"suites": [
{
"id": "fa02e22ab637686d665de5aeef0575ed",
"name": "mabl > ログイン",
"persistSession": false,
"parallel": false,
"timeout": 300,
"tests": [
"852afcfdae653511fe5ecdf7ae4fd8d9"
]
}
],
"id": "c4bd9d4f2dd42e3fe7e4b7f19bff29f5"
}.side はSelenium IDE形式。Selenium IDE形式などテストケースのエクスポートも簡単にできちゃう。そうmabl CLIならねを参考にどうぞ。
テスト結果一覧のダウンロード
"browser","status","failureReason","journey","labels","plan","environment","startedTime","runTime"
"chrome","completed","no failure reason available","Visit home page","default, smoke","Verify home page load and login - ホームページのロードとログイン (daipresents.com)","Production","Today at 10:51 PM","00:00:53"
"chrome","completed","no failure reason available","Visit home page","default, smoke","Verify home page load and login - ホームページのロードとログイン (daipresents.com)","Production","Today at 6:51 PM","00:00:44"
"chrome","completed","no failure reason available","Visit home page","default, smoke","Verify home page load and login - ホームページのロードとログイン (daipresents.com)","Production","Today at 2:51 PM","00:00:47"
"chrome","completed","no failure reason available","PDF download / PDFダウンロード","training","mabl sandbox plan","Production","Today at 1:00 PM","00:01:00"
"chrome","completed","no failure reason available","Email validation / Emailバリデーション","training","mabl sandbox plan","Production","Today at 1:00 PM","00:01:15"テスト結果
全体はPDF形式。各ステップはスクリーンショット、Stepトレース（Chromeの情報）、DOMスナップショット（HTML）、HTTP Archive（HAR)を取得できます。
// chrome_trace.json
{"traceEvents":[{"pid":336,"tid":336,"ts":3066165089,"ph":"(","cat":"blink","name":"FrameBlameContext","scope":"PlatformThread","id":"0x150","tts":38175,"args":{}},
{"pid":336,"tid":336,"ts":3066167089,"ph":"X","cat":"toplevel","name":"ThreadControllerImpl::RunTask","dur":41,"tdur":42,"tts":38244,"args":{"src_file":"../../ipc/ipc_mojo_bootstrap.cc","src_func":"Accept"}},
{"pid":336,"tid":336,"ts":3066167156,"ph":")","cat":"blink","name":"FrameBlameContext","scope":"PlatformThread","id":"0x150","tts":38312,"args":{}},
・・・// step.har
{"log":{"version":"1.2","creator":{"name":"runner","version":"15b0302","comment":""},"pages":[{"id":"Page 0","startedDateTime":"2020-03-01T13:51:25.681Z","title":"Page 0","pageTimings":{"onLoad":32026,"comment":""},"comment":""}],"entries":[{"pageref":"Page 0",
・・・CLIでmanifest.ymlとテスト結果のスクショをダウンロードできます。Planのテストまるごとダウンロードはできません。
% mabl test-runs export ${TEST_ID}
Export test run [${TEST_ID}]
✔ Creating export request [TF3Ls4kZ35ZSYffu1VcxSQ-ex]. Types to export: screenshots
✔ Processing export [TF3Ls4kZ35ZSYffu1VcxSQ-ex] status: [completed]
✔ Downloading export artifact to [TF3Ls4kZ35ZSYffu1VcxSQ-ex.zip]// manifest.yaml
export:
created_time: 1638339708569
entity_type: journey_run
export_types:
- !!io.longreen.api.v1.client.model.ExportRequest$TargetContentEnum 'SCREENSHOTS'
browser_type: chrome
journey_run_id: hAW1z6879ohKyQpDXFUyFA-jr
journey_id: jXCALY2kE0BldpXSbDaCkQ-j:10
browser_version: 96.0.4664.45
export_id: TF3Ls4kZ35ZSYffu1VcxSQ-exInsightをダウンロード
// Broken Linkの場合
"page","link","status"
"https://help.mabl.com/docs/mabl-best-practices#section-9-integrate-mabl-with-your-delivery-pipeline","https://plugins.jenkins.io/mabl-integration","Newly Broken"
"https://help.mabl.com/docs/frequently-asked-questions#section-can-mabl-run-on-premise-in-a-single-tenant-environment","https://plugins.jenkins.io/mabl-integration","Newly Broken"
・・・＊
CLIにはさまざまなコマンドがあるので、Environmentの一覧、Applicationの一覧、configの一覧なども出力できます。
Commands:
mabl applications <command> Manage your applications
mabl auth <command> Manage auth for the mabl CLI
mabl branches <command> Manage mabl branches
mabl config <command> Configure defaults for the mabl CLI
mabl credentials <command> Manage your testing credentials
mabl deployments <command> Manage mabl deployments
mabl environments <command> Manage your environments
mabl flows <command> Manage flows
mabl link-agents <command> Manage your link agents
mabl plans <command> Manage plans
mabl tests <command> Manage tests
mabl test-runs <command> Manage existing test runs
mabl workspaces <command> Manage your workspaces
旅と子育てとアジャイルコーチのブログ「世界」 