Introduction
Functionize offers the ability to attach an extension to run before a test starts or after it ends. Post-hook extensions are used at the end of a test to integrate with test management platforms or other services while having access to overall test and step-by-step results
- Go to the Setting icon, then click on the Test setting from the options list
Post-test Extensions
This is done in the Test Settings modal on the Webhooks tab. From here, you can select a Post-Test Extension. You can choose from any created extension that is available for your personal account or shared across your entire team, and a search field is provided to easily find the extension you are looking for. Once added, the extension will be visible in both the Test Settings -> Webhooks section, as well as the main Test Details page.
Note: An upgraded runtime version is necessary to enable this feature. Please contact support if you need to upgrade your runtime version.
Project Level Extensions
Extensions can also be attached at the project level, via the Webhooktab in theProject Settingsmodal. Any extension attached at the project level will execute for all tests in that project
Post-test Extensions
Here is an example request sent from Functionize into the extension:
{
"testResults": {
"result": "PASS",
"folderPath": "test-900000/job-10163538/"
},
"variables": {
"orchestration": {},
"project": {
"sampleObject": {
"testing": "123"
},
"sampleString": "value",
"SampleArray": [
"test",
"value"
]
},
"iteration": 0,
"team": {
"email": "tester@functionizeapp.com"
},
"tdm": {},
"env": {
"jobId": "10163538",
"testId": "900000",
"projectId": "24522",
"environment": "live",
"initialUrl": "https://www.google.com/",
"projSettingUrl": "google.com",
"timeouts": {
"pageLoadTimeout": 30,
"ajaxTimeout": 10,
"missingElementTimeout": 15
}
},
"local": {
"sampleLocal": {
"testing": "12345"
},
"iterations": [
0
]
},
"environment": "live",
"proxyIp": "10.200.1.20",
"orchestrationId": "0",
"testId": "900000",
"actionResults": [
{
"actionId": "1681486865296_V1CIIJZCL0",
"stepResult": {
"results": [
{
"result": "SUCCESS",
"message": ""
}
],
"executorResults": [
{
"executionMethodName": "pageinitMethod",
"successful": true,
"evaluatorResults": [
{
"evaluatorName": "empty",
"score": 100.0
}
],
"threshold": 0.0,
"confidenceScore": 100.0,
"totalEvals": 1.0,
"actionResults": {
"results": [
{
"result": "SUCCESS",
"message": ""
}
]
}
}
],
"actionMessages": [],
"resultCode": "PASS"
}
},
{
"actionId": "1681486871246_JYU07L63KF",
"stepResult": {
"results": [
{
"result": "SUCCESS",
"message": ""
}
],
"executorResults": [
{
"executionMethodName": "verifyAttributes",
"successful": true,
"evaluatorResults": [
{
"evaluatorName": "empty",
"score": 100.0
}
],
"threshold": 0.0,
"confidenceScore": 100.0,
"totalEvals": 1.0,
"actionResults": {
"results": [
{
"result": "SUCCESS",
"message": ""
}
]
}
}
],
"actionMessages": [],
"resultCode": "PASS"
}
}
],
"orchestrationRunId": "",
"TDMdataSetId": "",
"urlOverrideFrom": ""
}
}
The important fields to note:
testResults["result"]
- "PASS" - The test is in green and passes successfully
- "FAIL" - The test is red and one or more steps fail
- "WARNING" - The The test is orange and the test passes but some steps have critical warning
- "ERROR" - The test goes incomplete due to error
variables
- project - available across all tests in the project
- iteration - used for looping to keep track of iterations
- team - team level settings
- tdm - variables used for Test Data Management(TDM) execution
- env - environment variables
- local - available for this specific test
For more information, see https://support.functionize.com/hc/en-us/articles/4411989719575-Expressions-and-Variables
actionResults
This is a list of all actions executed in a test, in order, so the first item in the array will correspond to the first action in the test. This contains information on an action-by-action basis regarding the results of each action.
- action Messages - any message attached to the action describing the failure or warning
- resultCode - similar to testResults["result"] it can be "PASS", "FAIL", "WARNING", "Or ERROR", and for action level, it can also be "SUPPRESS" if the action is skipped.
- stepResult and executorResult - give some information about internal runtime processes. Only in very advanced use cases would these be needed.
Post Orchestration Extensions
Functionize offers to attach an extension to an orchestration as a Post-hook that will execute after the orchestration ends.
Here is a sample request that would be sent from FZE to the extension:
{'orchestration_id': '31839', 'title': 'APP Runtime Smoke Suite', 'runid': 'ab034f5217202f0c3c7cfeae459a6744', 'Status': 'Completed', 'total_testcases': 42, 'start_ts': '2024-02-08 21:12:15', 'end_ts': '2024-02-08 21:19:04', 'last_execution_status': 'Passed', 'warning': '1', 'passed': '41', 'failure': 0, 'tests': [{'status': 'Passed', 'job_id': '13769733', 'recording_id': '622269', 'name': 'API Test_Basic', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '96826'}, {'status': 'Passed', 'job_id': '13769745', 'recording_id': '636555', 'name': 'Architect Regression', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '91932'}, {'status': 'Passed', 'job_id': '13769736', 'recording_id': '649812', 'name': 'Architect Wildcard 2_v1.2.76', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '32018'}, {'status': 'Passed', 'job_id': '13769746', 'recording_id': '782484', 'name': 'Browser Back Forward Navigate', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '47233'}, {'status': 'Passed', 'job_id': '13769742', 'recording_id': '785814', 'name': 'Clone_Content Edible', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '37590'}, {'status': 'Passed', 'job_id': '13769740', 'recording_id': '782681', 'name': 'Clone_Demo - Multi-Tab Example_Amgen', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '50231'}, {'status': 'Passed', 'job_id': '13769744', 'recording_id': '782481', 'name': 'Clone_Hash Issue', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '36108'}, {'status': 'Passed', 'job_id': '13769747', 'recording_id': '782482', 'name': 'Clone_Language Drop-down List', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '34240'}, {'status': 'Passed', 'job_id': '13769741', 'recording_id': '782693', 'name': 'Clone_Page Variables', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '24851'}, {'status': 'Passed', 'job_id': '13769735', 'recording_id': '782684', 'name': 'Clone_Set Cookies & Set Local Storage', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '36652'}, {'status': 'Passed', 'job_id': '13769749', 'recording_id': '782685', 'name': 'Clone_Tooltip', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '45133'}, {'status': 'Passed', 'job_id': '13769756', 'recording_id': '622268', 'name': 'DB Query - Basic', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '60920'}, {'status': 'Passed', 'job_id': '13769753', 'recording_id': '627656', 'name': 'DB Query - Multi-TAB - Save Variable', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '59722'}, {'status': 'Passed', 'job_id': '13769738', 'recording_id': '627662', 'name': 'DB Query - Save Variable', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '53245'}, {'status': 'Passed', 'job_id': '13769750', 'recording_id': '782655', 'name': 'DB Query_Load Variable Verify On UI', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '36540'}, {'status': 'Passed', 'job_id': '13769734', 'recording_id': '718502', 'name': 'Demo - Click First Item In Search Results', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '44575'}, {'status': 'Passed', 'job_id': '13769757', 'recording_id': '718503', 'name': 'Demo - FileViewer_PDF', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '47026'}, {'status': 'Passed', 'job_id': '13769739', 'recording_id': '718500', 'name': 'Demo - Verify Search Results', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '29976'}, {'status': 'Passed', 'job_id': '13769751', 'recording_id': '718504', 'name': 'Demo - Visual Check', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '34629'}, {'status': 'Passed', 'job_id': '13769748', 'recording_id': '782486', 'name': 'Double Search & Text Select', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '51614'}, {'status': 'Passed', 'job_id': '13769754', 'recording_id': '782492', 'name': 'Email Reader', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '42677'}, {'status': 'Passed', 'job_id': '13769737', 'recording_id': '622264', 'name': 'FileViewer_Excel', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '38884'}, {'status': 'Passed', 'job_id': '13769755', 'recording_id': '794752', 'name': 'Functionize Internal Site with Accordion and SVG Elements', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '74319'}, {'status': 'Passed', 'job_id': '13769752', 'recording_id': '794753', 'name': 'Functionize Internal Site with SVG Elements', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '56734'}, {'status': 'Passed', 'job_id': '13769743', 'recording_id': '622263', 'name': 'FZ E-Comm Site_Architect with Instructions', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '39274'}, {'status': 'Passed', 'job_id': '13769774', 'recording_id': '643880', 'name': 'GenerateData Example', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '68529'}, {'status': 'Passed', 'job_id': '13769763', 'recording_id': '785773', 'name': 'iFrame Context Switch', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '170589'}, {'status': 'Passed', 'job_id': '13769771', 'recording_id': '783082', 'name': 'INPUT and TEXTSELECT', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '63181'}, {'status': 'Passed', 'job_id': '13769776', 'recording_id': '785833', 'name': 'Input Foreign Characters', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '41576'}, {'status': 'Passed', 'job_id': '13769770', 'recording_id': '785824', 'name': 'Input Single Character', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '37239'}, {'status': 'Passed', 'job_id': '13769769', 'recording_id': '783084', 'name': 'Maximize Window TC', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '36535'}, {'status': 'Passed', 'job_id': '13769761', 'recording_id': '782488', 'name': 'Multi-TAB Example_Functionize Site', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '45223'}, {'status': 'Passed', 'job_id': '13769766', 'recording_id': '786549', 'name': 'nested child frame.', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '59295'}, {'status': 'Passed', 'job_id': '13769762', 'recording_id': '782699', 'name': 'PO Parameters', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '42082'}, {'status': 'Passed', 'job_id': '13769760', 'recording_id': '643879', 'name': 'Resource Variable-Status Code', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '28372'}, {'status': 'Passed', 'job_id': '13769768', 'recording_id': '782490', 'name': 'Select & Random Email', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '45782'}, {'status': 'Passed', 'job_id': '13769764', 'recording_id': '658667', 'name': 'Self-Heal Tags Buttons-to-List', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '32491'}, {'status': 'Warning', 'job_id': '13769773', 'recording_id': '782619', 'name': 'Test Most Actions', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '267611'}, {'status': 'Passed', 'job_id': '13769775', 'recording_id': '634394', 'name': 'Verify Previous Step', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '44096'}, {'status': 'Passed', 'job_id': '13769767', 'recording_id': '635232', 'name': 'Verify with Variable', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '48887'}, {'status': 'Passed', 'job_id': '13769765', 'recording_id': '643881', 'name': 'Google Search TDM', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '44997'}, {'status': 'Passed', 'job_id': '13769772', 'recording_id': '1074117', 'name': 'Upload pdf', 'browser_name': 'Chrome 110', 'DataSetName': '', 'duration': '86605'}], 'userAccessToken': '2bd1d09bbdab3f35b9d03f14d95d046e239cdc86'}
- Click on Edit Orchestrations.
- Edit Orchestration Window will open, Click on Notification and then click on Webhooks.
- Click on Add Extension select the Extension, Fill the Post hook Proxy If required, and Click Confirm.