スポンサーリンク

mablからダウンロード/エクスポートできるものを調べてみた #mabljapan

Selenium IDE exportに「not worry about vendor lock-in when using mabl.」とあるように、mablは基本的にいろんなものをダウンロード/エクスポートできます。それぞれどういうものかをみてみましょう。基本CSV形式です。

スポンサーリンク

Planのダウンロード

"APPLICATION_ID","APPLICATION_NAME","DEPLOYMENT_ID","PLAN_ID","PLAN_NAME","PLAN_RUN_ID","PLAN_URL","RUNTIME_MILLIS","STARTING_URL","START_TIME","STATUS","STOP_TIME","SUCCESS","TAGS","TRIGGER_TYPE","TRIGGER_USER_EMAIL","TRIGGER_USER_ID","WORKSPACE_ID","WORKSPACE_NAME"
"XXXX","mabl sandbox","YYYY","ZZZZ","mabl  sandbox plan","AAAAA","https://app.mabl.com/FFFFFF/test/plans/AAAA","81306","https://sandbox.mabl.com/","1583035204303","succeeded","1583035285609","false","","mabl_scheduler","","v1_8","GGGGG","daipresents workspace"
"XXXX","mabl sandbox","YYYY","ZZZZ","mabl  sandbox plan","BBBBB","https://app.mabl.com/FFFFFF/test/plans/AAAA","74473","https://sandbox.mabl.com/","1582948764871","succeeded","1582948839344","false","","mabl_scheduler","daipresents","GGGGG","EEEE","daipresents workspace"

Plan一覧のダウンロード

CLIを利用。

% mabl plans list --limit 1
┌──────────────────────────┬─────────────────────────────────┬─────────────────────────┐
│ ID                       │ Name                            │ Created time            │
├──────────────────────────┼─────────────────────────────────┼─────────────────────────┤
│ Ccix0Meyd8znbQ2PK80Q6w-p │ Verify home page load and login │ Aug 11th 2021, 07:12:01 │
└──────────────────────────┴─────────────────────────────────┴─────────────────────────┘
1 plans returned

テストケース一覧のダウンロード

"id","journey","percentagePassed","planCount","createdBy","createdDate","description"
"XXXXXXXXXXXXXXXXXXX","Custom HTTP Header の確認","100","1","Dai Fujihara","Feb 10, 2020","Custom HTTP Headerの確認。JSからのアクセス(navigator.userAgent)と HTTPヘッダの中身を確認しています。
参考:https://daipresents.com/2020/creating-mobile-web-tests/"
"YYYYYYYYYYYYYYYYYYY","API request and response validation / APIのテスト","100","1","Dai Fujihara","Jan 30, 2020","APIを叩いてそのレスポンスをチェックしています。これによってテストデータをAPI経由で準備したり、データの状態を変更してテストを進められるためテストが簡潔になります。
参考: https://daipresents.com/2020/mabl-api-requests-response-validation/"
"ZZZZZZZZZZZZZZZZZZZZZ","Mobile View - モバイルページの確認","100","1","Dai Fujihara","Jan 29, 2020","レスポンシブルページをモバイルブラウザで表示します。mablではCustom HTTP HeaderやViewportの設定が可能です。
参考:https://daipresents.com/2020/creating-mobile-web-tests/
"
"DEFAULTTTTTTTTTTTTTTT","Visit all linked pages within the app","70","3","Dai Fujihara","Jan 7, 2020","Starting at the primary url, crawl all links within the application's domain, validate that the linked pages load."
"DEFAULTTTTTTTTTTTTTTT","Visit home page","100","3","Dai Fujihara","Jan 7, 2020","Attempt to load the primary url in a browser, validate that it loads, verify that login works (if provided)."

journeyはTestを指している。昔、mablはTestをJourneyと呼んでいたそう。

テストケースのダウンロード(CSV)

 ,Step Number, Step Type, Description, Annotation, Note
0, Echo, Echo: "# ログイン(成功)の観点をチェック", -, -
1, Echo, Echo: "## 共通処理をここで定義", -, -
2, VisitUrl, Visit URL "{{@app.url}}", -, "{{@web.defaults.url}}"を開く
3, SetViewport, Set viewport size to width 1600, -, 画面の幅を1600pxに設定
4, Echo, Echo: "## ログイン処理", -, -
5, Click, Click on the <span> element with text "SIMULATED LOGIN", -, "SIMULATED LOGIN"をクリックする
6, WaitUntil, Wait up to 15 seconds until target element matching Text attribute is present, -, ログインボタンが表示されることまでまつ
7, EnterText, Insert value of variable "flow.email" into the "email" text field, -, Emailを入力
8, EnterText, Insert value of variable "flow.password" into the "password" text field, -, パスワードを入力
9, Click, Click on the "LOG IN" button, -, ログインボタンをクリック
10, WaitUntil, Wait up to 15 seconds until target element matching Text attribute is present, -, 次の画面が表示されるまで待つ
11, Echo, Echo: "## ログイン成功時の確認", -, -
12, AssertPresent, Assert presence of the <span> element with text "LOG OUT", -, "LOG OUT"ボタンが表示されていることを確認

テストケースのダウンロード(JSON)

{
  "id": "jXCALY2kE0BldpXSbDaCkQ-j:10",
  "name": "mabl > ログイン",
  "description": "",
  "steps": [
    {
      "Echo": {
        "description": "Echo: \"# ログイン(成功)の観点をチェック\""
      }
    },
    {
      "Echo": {
        "description": "Echo: \"## 共通処理をここで定義\""
      }
    },
    {
      "VisitUrl": {
        "description": "Visit URL \"{{@app.url}}\"",
        "annotation": {
          "description": "\"{{@web.defaults.url}}\"を開く"
        }
      }
    },
    {
      "SetViewport": {
        "description": "Set viewport size to width 1600",
        "annotation": {
          "description": "画面の幅を1600pxに設定"
        }
      }
    },
    {
      "Echo": {
        "description": "Echo: \"## ログイン処理\""
      }
    },
    {
      "Click": {
        "description": "Click on the <span> element with text \"SIMULATED LOGIN\"",
        "annotation": {
          "description": "\"SIMULATED LOGIN\"をクリックする"
        }
      }
    },
    {
      "WaitUntil": {
        "description": "Wait up to 15 seconds until target element matching Text attribute is present",
        "annotation": {
          "description": "ログインボタンが表示されることまでまつ"
        }
      }
    },
    {
      "EnterText": {
        "description": "Insert value of variable \"flow.email\" into the \"email\" text field",
        "annotation": {
          "description": "Emailを入力"
        }
      }
    },
    {
      "EnterText": {
        "description": "Insert value of variable \"flow.password\" into the \"password\" text field",
        "annotation": {
          "description": "パスワードを入力"
        }
      }
    },
    {
      "Click": {
        "description": "Click on the \"LOG IN\" button",
        "annotation": {
          "description": "ログインボタンをクリック"
        }
      }
    },
    {
      "WaitUntil": {
        "description": "Wait up to 15 seconds until target element matching Text attribute is present",
        "annotation": {
          "description": "次の画面が表示されるまで待つ"
        }
      }
    },
    {
      "Echo": {
        "description": "Echo: \"## ログイン成功時の確認\""
      }
    },
    {
      "AssertPresent": {
        "description": "Assert presence of the <span> element with text \"LOG OUT\"",
        "annotation": {
          "description": "\"LOG OUT\"ボタンが表示されていることを確認"
        }
      }
    }
  ]
}

テストケースのダウンロード(YAML)

id: 'jXCALY2kE0BldpXSbDaCkQ-j:10'
name: mabl > ログイン
description: ''
steps:
  - Echo:
      description: 'Echo: "# ログイン(成功)の観点をチェック"'
  - Echo:
      description: 'Echo: "## 共通処理をここで定義"'
  - VisitUrl:
      description: 'Visit URL "{{@app.url}}"'
      annotation:
        description: '"{{@web.defaults.url}}"を開く'
  - SetViewport:
      description: Set viewport size to width 1600
      annotation:
        description: 画面の幅を1600pxに設定
  - Echo:
      description: 'Echo: "## ログイン処理"'
  - Click:
      description: Click on the <span> element with text "SIMULATED LOGIN"
      annotation:
        description: '"SIMULATED LOGIN"をクリックする'
  - WaitUntil:
      description: >-
        Wait up to 15 seconds until target element matching Text attribute is
        present
      annotation:
        description: ログインボタンが表示されることまでまつ
  - EnterText:
      description: Insert value of variable "flow.email" into the "email" text field
      annotation:
        description: Emailを入力
  - EnterText:
      description: Insert value of variable "flow.password" into the "password" text field
      annotation:
        description: パスワードを入力
  - Click:
      description: Click on the "LOG IN" button
      annotation:
        description: ログインボタンをクリック
  - WaitUntil:
      description: >-
        Wait up to 15 seconds until target element matching Text attribute is
        present
      annotation:
        description: 次の画面が表示されるまで待つ
  - Echo:
      description: 'Echo: "## ログイン成功時の確認"'
  - AssertPresent:
      description: Assert presence of the <span> element with text "LOG OUT"
      annotation:
        description: '"LOG OUT"ボタンが表示されていることを確認'

テストケースのダウンロード(SIDE)

{
  "url": "https://sandbox.mabl.com/",
  "version": "2.0",
  "name": "mabl > ログイン",
  "tests": [
    {
      "id": "852afcfdae653511fe5ecdf7ae4fd8d9",
      "name": "mabl > ログイン",
      "commands": [
        {
          "id": "3accba5308ecc608c14f3beb3b71a31b",
          "comment": "",
          "command": "store",
          "target": "https://sandbox.mabl.com/",
          "targets": [],
          "value": "web.defaults.url"
        },
        {
          "id": "4e65700548b37924cad1e8c1d2a1e0f5",
          "comment": "Echo: \"# ログイン(成功)の観点をチェック\"",
          "command": "echo",
          "target": "# ログイン(成功)の観点をチェック",
          "targets": [],
          "value": ""
        },
        {
          "id": "261ff28e12d9d8acc2d012b49c79e8c3",
          "comment": "Echo: \"## 共通処理をここで定義\"",
          "command": "echo",
          "target": "## 共通処理をここで定義",
          "targets": [],
          "value": ""
        },
        {
          "id": "ebe1742ae090ff8809f52d512df4d939",
          "comment": "Visit URL \"{{@app.url}}\"",
          "command": "open",
          "target": "${web.defaults.url}",
          "targets": [],
          "value": ""
        },
        {
          "id": "22bb3687d905a457d901243959414403",
          "comment": "Set viewport size to width 1600",
          "command": "setWindowSize",
          "target": "1600x1080",
          "targets": [],
          "value": ""
        },
        {
          "id": "dc2990aa43042f3997af77fc99c5095c",
          "comment": "Echo: \"## ログイン処理\"",
          "command": "echo",
          "target": "## ログイン処理",
          "targets": [],
          "value": ""
        },
        {
          "id": "e81c1abb311267844c2f61fd4ac8c7cd",
          "comment": "Click on the <span> element with text \"SIMULATED LOGIN\"",
          "command": "click",
          "target": "xpath=//html[1]/body[1]/div[1]/div[2]/div[1]/div[1]/div[1]/div[2]/div[1]/a[13]/span[1]",
          "targets": [],
          "value": ""
        },
        {
          "id": "bd7fdc2b707135bedf1e3ec06d6ce6de",
          "comment": "Wait up to 15 seconds until target element matching Text attribute is present",
          "command": "assertElementPresent",
          "target": "xpath=//html[1]/body[1]/div[1]/div[2]/div[1]/div[2]/form[1]/button[1]/span[1]",
          "targets": [],
          "value": ""
        },
        {
          "id": "6b5e30d76f2c332d120fce217f5e6982",
          "comment": "Insert value of variable \"flow.email\" into the \"email\" text field",
          "command": "type",
          "target": "xpath=//html[1]/body[1]/div[1]/div[2]/div[1]/div[2]/form[1]/div[1]/div[1]/input[1]",
          "targets": [],
          "value": "${flow.email}"
        },
        {
          "id": "c6389c7116f612555a2e0bcfbd3be920",
          "comment": "Insert value of variable \"flow.password\" into the \"password\" text field",
          "command": "type",
          "target": "xpath=//html[1]/body[1]/div[1]/div[2]/div[1]/div[2]/form[1]/div[2]/div[1]/input[1]",
          "targets": [],
          "value": "${flow.password}"
        },
        {
          "id": "8b46450db11f30da0957aa8230696af3",
          "comment": "Click on the \"LOG IN\" button",
          "command": "click",
          "target": "xpath=//html[1]/body[1]/div[1]/div[2]/div[1]/div[2]/form[1]/button[1]",
          "targets": [],
          "value": ""
        },
        {
          "id": "bd7fdc2b707135bedf1e3ec06d6ce6de",
          "comment": "Wait up to 15 seconds until target element matching Text attribute is present",
          "command": "assertElementPresent",
          "target": "xpath=//html[1]/body[1]/div[1]/div[2]/div[1]/div[2]/button[1]/span[1]",
          "targets": [],
          "value": ""
        },
        {
          "id": "00038cde5e49ac1c9bfb609875a29b70",
          "comment": "Echo: \"## ログイン成功時の確認\"",
          "command": "echo",
          "target": "## ログイン成功時の確認",
          "targets": [],
          "value": ""
        },
        {
          "id": "d2f1194a2bc834da5a43142b3ecd9f11",
          "comment": "Assert presence of the <span> element with text \"LOG OUT\"",
          "command": "assertElementPresent",
          "target": "xpath=//html[1]/body[1]/div[1]/div[2]/div[1]/div[2]/button[1]/span[1]",
          "targets": [],
          "value": ""
        }
      ]
    }
  ],
  "suites": [
    {
      "id": "fa02e22ab637686d665de5aeef0575ed",
      "name": "mabl > ログイン",
      "persistSession": false,
      "parallel": false,
      "timeout": 300,
      "tests": [
        "852afcfdae653511fe5ecdf7ae4fd8d9"
      ]
    }
  ],
  "id": "c4bd9d4f2dd42e3fe7e4b7f19bff29f5"
}

.side はSelenium IDE形式。Selenium IDE形式などテストケースのエクスポートも簡単にできちゃう。そうmabl CLIならねを参考にどうぞ。

テスト結果一覧のダウンロード

"browser","status","failureReason","journey","labels","plan","environment","startedTime","runTime"
"chrome","completed","no failure reason available","Visit home page","default, smoke","Verify home page load and login - ホームページのロードとログイン (daipresents.com)","Production","Today at 10:51 PM","00:00:53"
"chrome","completed","no failure reason available","Visit home page","default, smoke","Verify home page load and login - ホームページのロードとログイン (daipresents.com)","Production","Today at 6:51 PM","00:00:44"
"chrome","completed","no failure reason available","Visit home page","default, smoke","Verify home page load and login - ホームページのロードとログイン (daipresents.com)","Production","Today at 2:51 PM","00:00:47"
"chrome","completed","no failure reason available","PDF download  / PDFダウンロード","training","mabl  sandbox plan","Production","Today at 1:00 PM","00:01:00"
"chrome","completed","no failure reason available","Email validation / Emailバリデーション","training","mabl  sandbox plan","Production","Today at 1:00 PM","00:01:15"

テスト結果

全体はPDF形式。各ステップはスクリーンショット、Stepトレース(Chromeの情報)、DOMスナップショット(HTML)、HTTP Archive(HAR)を取得できます。

PDF形式
// chrome_trace.json
{"traceEvents":[{"pid":336,"tid":336,"ts":3066165089,"ph":"(","cat":"blink","name":"FrameBlameContext","scope":"PlatformThread","id":"0x150","tts":38175,"args":{}},
{"pid":336,"tid":336,"ts":3066167089,"ph":"X","cat":"toplevel","name":"ThreadControllerImpl::RunTask","dur":41,"tdur":42,"tts":38244,"args":{"src_file":"../../ipc/ipc_mojo_bootstrap.cc","src_func":"Accept"}},
{"pid":336,"tid":336,"ts":3066167156,"ph":")","cat":"blink","name":"FrameBlameContext","scope":"PlatformThread","id":"0x150","tts":38312,"args":{}},
・・・
// step.har
{"log":{"version":"1.2","creator":{"name":"runner","version":"15b0302","comment":""},"pages":[{"id":"Page 0","startedDateTime":"2020-03-01T13:51:25.681Z","title":"Page 0","pageTimings":{"onLoad":32026,"comment":""},"comment":""}],"entries":[{"pageref":"Page 0",
・・・

CLIでmanifest.ymlとテスト結果のスクショをダウンロードできます。Planのテストまるごとダウンロードはできません。

 % mabl test-runs export ${TEST_ID}
Export test run [${TEST_ID}]
  ✔ Creating export request [TF3Ls4kZ35ZSYffu1VcxSQ-ex].  Types to export: screenshots
  ✔ Processing export [TF3Ls4kZ35ZSYffu1VcxSQ-ex] status: [completed]
  ✔ Downloading export artifact to [TF3Ls4kZ35ZSYffu1VcxSQ-ex.zip]
// manifest.yaml
export:
  created_time: 1638339708569
  entity_type: journey_run
  export_types:
  - !!io.longreen.api.v1.client.model.ExportRequest$TargetContentEnum 'SCREENSHOTS'
  browser_type: chrome
  journey_run_id: hAW1z6879ohKyQpDXFUyFA-jr
  journey_id: jXCALY2kE0BldpXSbDaCkQ-j:10
  browser_version: 96.0.4664.45
  export_id: TF3Ls4kZ35ZSYffu1VcxSQ-ex

Insightをダウンロード

// Broken Linkの場合
"page","link","status"
"https://help.mabl.com/docs/mabl-best-practices#section-9-integrate-mabl-with-your-delivery-pipeline","https://plugins.jenkins.io/mabl-integration","Newly Broken"
"https://help.mabl.com/docs/frequently-asked-questions#section-can-mabl-run-on-premise-in-a-single-tenant-environment","https://plugins.jenkins.io/mabl-integration","Newly Broken"
・・・

CLIにはさまざまなコマンドがあるので、Environmentの一覧、Applicationの一覧、configの一覧なども出力できます。

Commands:
  mabl applications <command>  Manage your applications
  mabl auth <command>          Manage auth for the mabl CLI
  mabl branches <command>      Manage mabl branches
  mabl config <command>        Configure defaults for the mabl CLI
  mabl credentials <command>   Manage your testing credentials
  mabl deployments <command>   Manage mabl deployments
  mabl environments <command>  Manage your environments
  mabl flows <command>         Manage flows
  mabl link-agents <command>   Manage your link agents
  mabl plans <command>         Manage plans
  mabl tests <command>         Manage tests
  mabl test-runs <command>     Manage existing test runs
  mabl workspaces <command>    Manage your workspaces
タイトルとURLをコピーしました