Skip to content

Commit 849b05a

Browse files
authored
Merge pull request #2636 from OwenK2/2634-scenario-outline-metadata
Add afterScenarioOutline & karate.scenarioOutline
2 parents 887c33a + 0da7b56 commit 849b05a

21 files changed

+500
-10
lines changed

README.md

+3
Original file line numberDiff line numberDiff line change
@@ -2252,6 +2252,7 @@ You can adjust configuration settings for the HTTP client used by Karate using t
22522252
`printEnabled` | boolean | Can be used to suppress the [`print`](#print) output when not in 'dev mode' by setting as `false` (default `true`)
22532253
`report` | JSON / boolean | see [report verbosity](#report-verbosity)
22542254
`afterScenario` | JS function | Will be called [after every `Scenario`](#hooks) (or `Example` within a `Scenario Outline`), refer to this example: [`hooks.feature`](karate-demo/src/test/java/demo/hooks/hooks.feature)
2255+
`afterScenarioOutline` | JS function | Will be called [after every `Scenario Outline`](#hooks). Is called after the last `afterScenario` for the last scenario in the outline. Refer to this example: [`hooks.feature`](karate-demo/src/test/java/demo/hooks/hooks.feature)
22552256
`afterFeature` | JS function | Will be called [after every `Feature`](#hooks), refer to this example: [`hooks.feature`](karate-demo/src/test/java/demo/hooks/hooks.feature)
22562257
`ssl` | boolean | Enable HTTPS calls without needing to configure a trusted certificate or key-store.
22572258
`ssl` | string | Like above, but force the SSL algorithm to one of [these values](http://docs.oracle.com/javase/8/docs/technotes/guides/security/StandardNames.html#SSLContext). (The above form internally defaults to `TLS` if simply set to `true`).
@@ -3690,6 +3691,7 @@ Operation | Description
36903691
<a name="karate-response"><code>karate.response</code></a> | returns the last HTTP response as a JS object that enables advanced use-cases such as getting a header ignoring case: `karate.response.header('some-header')`
36913692
<a name="karate-request"><code>karate.request</code></a> | returns the last HTTP request as a JS object that enables advanced use-cases such as getting a header ignoring case: `karate.request.header('some-header')`, which works [even in mocks](https://github.com/karatelabs/karate/tree/master/karate-netty#requestheaders)
36923693
<a name="karate-scenario"><code>karate.scenario</code></a> | get metadata about the currently executing `Scenario` (or `Outline` - `Example`) within a test
3694+
<a name="karate-scenarioOutline"><code>karate.scenarioOutline</code></a> | get metadata about the currently executing scenario outline within a test
36933695
<a name="karate-set"><code>karate.set(name, value)</code></a> | sets the value of a variable (immediately), which may be needed in case any other routines (such as the [configured headers](#configure-headers)) depend on that variable
36943696
<a name="karate-setall"><code>karate.set(object)</code></a> | where the single argument is expected to be a `Map` or JSON-like, and will perform the above `karate.set()` operation for all key-value pairs in one-shot
36953697
<a name="karate-setpath"><code>karate.set(name, path, value)</code></a> | only needed when you need to conditionally build payload elements, especially XML. This is best explained via [an example](karate-core/src/test/java/com/intuit/karate/core/xml/xml.feature#L211), and it behaves the same way as the [`set`](#set) keyword. Also see [`eval`](#eval).
@@ -4440,6 +4442,7 @@ Before *everything* (or 'globally' once) | See [`karate.callSingle()`](#karateca
44404442
Before every `Scenario` | Use the [`Background`](#script-structure). Note that [`karate-config.js`](#karate-configjs) is processed before *every* `Scenario` - so you can choose to put "global" config here, for example using [`karate.configure()`](#karate-configure).
44414443
Once (or at the start of) every `Feature` | Use a [`callonce`](#callonce) in the [`Background`](#script-structure). The advantage is that you can set up variables (using [`def`](#def) if needed) which can be used in all `Scenario`-s within that `Feature`.
44424444
After every `Scenario` | [`configure afterScenario`](#configure) (see [example](karate-demo/src/test/java/demo/hooks/hooks.feature))
4445+
After every `Scenario Outline` | [`configure afterScenarioOutline`](#configure) (see [example](karate-demo/src/test/java/demo/hooks/hooks.feature))
44434446
At the end of the `Feature` | [`configure afterFeature`](#configure) (see [example](karate-demo/src/test/java/demo/hooks/hooks.feature))
44444447

44454448
> Note that for the `afterFeature` hook to work, you should be using the [`Runner` API](#parallel-execution) and not the JUnit runner.

karate-core/src/main/java/com/intuit/karate/RuntimeHook.java

+4
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,10 @@ default void afterScenario(ScenarioRuntime sr) {
4646

4747
}
4848

49+
default void afterScenarioOutline(ScenarioRuntime sr) {
50+
51+
}
52+
4953
default boolean beforeFeature(FeatureRuntime fr) {
5054
return true;
5155
}
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
/*
2+
* The MIT License
3+
*
4+
* Copyright 2022 Karate Labs Inc.
5+
*
6+
* Permission is hereby granted, free of charge, to any person obtaining a copy
7+
* of this software and associated documentation files (the "Software"), to deal
8+
* in the Software without restriction, including without limitation the rights
9+
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
10+
* copies of the Software, and to permit persons to whom the Software is
11+
* furnished to do so, subject to the following conditions:
12+
*
13+
* The above copyright notice and this permission notice shall be included in
14+
* all copies or substantial portions of the Software.
15+
*
16+
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
17+
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
18+
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
19+
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
20+
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
21+
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
22+
* THE SOFTWARE.
23+
*/
24+
package com.intuit.karate.core;
25+
26+
/**
27+
*
28+
* @author OwenK2
29+
*/
30+
public enum AfterHookType {
31+
32+
AFTER_SCENARIO("afterScenario"),
33+
AFTER_OUTLINE("afterScenarioOutline"),
34+
AFTER_FEATURE("afterFeature");
35+
36+
private String prefix;
37+
38+
private AfterHookType(String prefix) {
39+
this.prefix = prefix;
40+
}
41+
42+
public String getPrefix() {
43+
return prefix;
44+
}
45+
}

karate-core/src/main/java/com/intuit/karate/core/Config.java

+13
Original file line numberDiff line numberDiff line change
@@ -93,6 +93,7 @@ public class Config {
9393
private HttpLogModifier logModifier;
9494

9595
private Variable afterScenario = Variable.NULL;
96+
private Variable afterScenarioOutline = Variable.NULL;
9697
private Variable afterFeature = Variable.NULL;
9798
private Variable headers = Variable.NULL;
9899
private Variable cookies = Variable.NULL;
@@ -175,6 +176,9 @@ public boolean configure(String key, Variable value) { // TODO use enum
175176
case "afterScenario":
176177
afterScenario = value;
177178
return false;
179+
case "afterScenarioOutline":
180+
afterScenarioOutline = value;
181+
return false;
178182
case "afterFeature":
179183
afterFeature = value;
180184
return false;
@@ -382,6 +386,7 @@ public Config(Config parent) {
382386
cookies = parent.cookies;
383387
responseHeaders = parent.responseHeaders;
384388
afterScenario = parent.afterScenario;
389+
afterScenarioOutline = parent.afterScenarioOutline;
385390
afterFeature = parent.afterFeature;
386391
continueOnStepFailureMethods = parent.continueOnStepFailureMethods;
387392
continueAfterContinueOnStepFailure = parent.continueAfterContinueOnStepFailure;
@@ -538,6 +543,14 @@ public void setAfterScenario(Variable afterScenario) {
538543
this.afterScenario = afterScenario;
539544
}
540545

546+
public Variable getAfterScenarioOutline() {
547+
return afterScenarioOutline;
548+
}
549+
550+
public void setAfterScenarioOutline(Variable afterScenarioOutline) {
551+
this.afterScenarioOutline = afterScenarioOutline;
552+
}
553+
541554
public Variable getAfterFeature() {
542555
return afterFeature;
543556
}

karate-core/src/main/java/com/intuit/karate/core/ExamplesTable.java

+12
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,9 @@
2424
package com.intuit.karate.core;
2525

2626
import java.util.ArrayList;
27+
import java.util.HashMap;
2728
import java.util.List;
29+
import java.util.Map;
2830

2931
/**
3032
*
@@ -35,6 +37,7 @@ public class ExamplesTable {
3537
private final ScenarioOutline outline;
3638
private final Table table;
3739
private List<Tag> tags;
40+
3841

3942
public ExamplesTable(ScenarioOutline outline, Table table) {
4043
this.outline = outline;
@@ -58,4 +61,13 @@ public Table getTable() {
5861
return table;
5962
}
6063

64+
public Map<String, Object> toKarateJson() {
65+
Map<String, Object> map = new HashMap();
66+
List<String> tagStrings = new ArrayList();
67+
tags.forEach(tag -> tagStrings.add(tag.toString()));
68+
map.put("tags", tagStrings);
69+
map.put("data", table.getRowsAsMapsConverted());
70+
return map;
71+
}
72+
6173
}

karate-core/src/main/java/com/intuit/karate/core/FeatureRuntime.java

+23-1
Original file line numberDiff line numberDiff line change
@@ -195,16 +195,38 @@ private void processScenario(ScenarioRuntime sr) {
195195
if (!sr.result.getStepResults().isEmpty()) {
196196
synchronized (result) {
197197
result.addResult(sr.result);
198+
199+
// Execute afterScenarioOutline if applicable
200+
// NOTE: Needs to be run after adding result, since result count is used to deterime
201+
// if the scenario is the last in the outline
202+
if (!sr.dryRun && isLastScenarioInOutline(sr.scenario)) {
203+
sr.engine.invokeAfterHookIfConfigured(AfterHookType.AFTER_OUTLINE);
204+
suite.hooks.forEach(h -> h.afterScenarioOutline(sr));
205+
}
198206
}
199207
}
200208
}
201209
}
202210

211+
private boolean isLastScenarioInOutline(Scenario scenario) {
212+
// Check if scenario is part of an outline
213+
if (!scenario.isOutlineExample()) return false;
214+
215+
// Count the number of completed scenarios with the same section ID (in same outline)
216+
int completedScenarios = 0;
217+
for (ScenarioResult result : result.getScenarioResults()) {
218+
if (result.getScenario().getSection().getIndex() == scenario.getSection().getIndex()) {
219+
completedScenarios++;
220+
}
221+
}
222+
return completedScenarios == scenario.getSection().getScenarioOutline().getNumScenarios();
223+
}
224+
203225
// extracted for junit5
204226
public synchronized void afterFeature() {
205227
result.sortScenarioResults();
206228
if (lastExecutedScenario != null) {
207-
lastExecutedScenario.engine.invokeAfterHookIfConfigured(true);
229+
lastExecutedScenario.engine.invokeAfterHookIfConfigured(AfterHookType.AFTER_FEATURE);
208230
result.setVariables(lastExecutedScenario.engine.getAllVariablesAsMap());
209231
result.setConfig(lastExecutedScenario.engine.getConfig());
210232
}

karate-core/src/main/java/com/intuit/karate/core/ScenarioBridge.java

+4
Original file line numberDiff line numberDiff line change
@@ -551,6 +551,10 @@ public Object getScenario() {
551551
return new JsMap(getEngine().runtime.result.toKarateJson());
552552
}
553553

554+
public Object getScenarioOutline() {
555+
return new JsMap(getEngine().runtime.outlineResult.toKarateJson());
556+
}
557+
554558
public Object getTags() {
555559
return JsValue.fromJava(getEngine().runtime.tags.getTags());
556560
}

karate-core/src/main/java/com/intuit/karate/core/ScenarioEngine.java

+20-5
Original file line numberDiff line numberDiff line change
@@ -235,20 +235,35 @@ public void print(String exp) {
235235
evalJs("karate.log('[print]'," + exp + ")");
236236
}
237237

238-
public void invokeAfterHookIfConfigured(boolean afterFeature) {
238+
public void invokeAfterHookIfConfigured(AfterHookType hookType) {
239+
// Do not call hooks on "called" scenarios/features
239240
if (runtime.caller.depth > 0) {
240241
return;
241242
}
242-
Variable v = afterFeature ? config.getAfterFeature() : config.getAfterScenario();
243+
244+
// Get hook variable based on type
245+
Variable v;
246+
switch (hookType) {
247+
case AFTER_SCENARIO:
248+
v = config.getAfterScenario();
249+
break;
250+
case AFTER_OUTLINE:
251+
v = config.getAfterScenarioOutline();
252+
break;
253+
case AFTER_FEATURE:
254+
v = config.getAfterFeature();
255+
break;
256+
default: return;
257+
}
258+
243259
if (v.isJsOrJavaFunction()) {
244-
if (afterFeature) {
260+
if (hookType == AfterHookType.AFTER_FEATURE) {
245261
ScenarioEngine.set(this); // for any bridge / js to work
246262
}
247263
try {
248264
executeFunction(v);
249265
} catch (Exception e) {
250-
String prefix = afterFeature ? "afterFeature" : "afterScenario";
251-
logger.warn("{} hook failed: {}", prefix, e + "");
266+
logger.warn("{} hook failed: {}", hookType.getPrefix(), e + "");
252267
}
253268
}
254269
}

karate-core/src/main/java/com/intuit/karate/core/ScenarioOutline.java

+17
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@
2525

2626
import java.util.ArrayList;
2727
import java.util.List;
28+
import java.util.Map;
2829

2930
/**
3031
* @author pthomas3
@@ -40,6 +41,7 @@ public class ScenarioOutline {
4041
private String description;
4142
private List<Step> steps;
4243
private List<ExamplesTable> examplesTables;
44+
private int numScenarios = 0;
4345

4446
public ScenarioOutline(Feature feature, FeatureSection section) {
4547
this.feature = feature;
@@ -75,6 +77,7 @@ public Scenario toScenario(String dynamicExpression, int exampleIndex, int updat
7577
step.setTable(original.getTable());
7678
step.setComments(original.getComments());
7779
}
80+
numScenarios++;
7881
return s;
7982
}
8083

@@ -167,9 +170,23 @@ public void setSteps(List<Step> steps) {
167170
public List<ExamplesTable> getExamplesTables() {
168171
return examplesTables;
169172
}
173+
174+
public int getNumExampleTables() {
175+
return examplesTables.size();
176+
}
177+
178+
public List<Map<String, Object>> getAllExampleData() {
179+
List<Map<String, Object>> exampleData = new ArrayList();
180+
examplesTables.forEach(table -> exampleData.add(table.toKarateJson()));
181+
return exampleData;
182+
}
170183

171184
public void setExamplesTables(List<ExamplesTable> examplesTables) {
172185
this.examplesTables = examplesTables;
173186
}
187+
188+
public int getNumScenarios() {
189+
return numScenarios;
190+
}
174191

175192
}
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
/*
2+
* The MIT License
3+
*
4+
* Copyright 2022 Karate Labs Inc.
5+
*
6+
* Permission is hereby granted, free of charge, to any person obtaining a copy
7+
* of this software and associated documentation files (the "Software"), to deal
8+
* in the Software without restriction, including without limitation the rights
9+
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
10+
* copies of the Software, and to permit persons to whom the Software is
11+
* furnished to do so, subject to the following conditions:
12+
*
13+
* The above copyright notice and this permission notice shall be included in
14+
* all copies or substantial portions of the Software.
15+
*
16+
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
17+
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
18+
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
19+
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
20+
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
21+
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
22+
* THE SOFTWARE.
23+
*/
24+
package com.intuit.karate.core;
25+
26+
import java.util.ArrayList;
27+
import java.util.HashMap;
28+
import java.util.List;
29+
import java.util.Map;
30+
31+
/**
32+
*
33+
* @author OwenK2
34+
*/
35+
public class ScenarioOutlineResult {
36+
37+
final private ScenarioOutline scenarioOutline;
38+
final private ScenarioRuntime runtime;
39+
40+
public ScenarioOutlineResult(ScenarioOutline scenarioOutline, ScenarioRuntime runtime) {
41+
// NOTE: this value can be null, in which case the scenario is not from an outline
42+
this.scenarioOutline = scenarioOutline;
43+
this.runtime = runtime;
44+
}
45+
46+
public Map<String, Object> toKarateJson() {
47+
if (scenarioOutline == null) return null;
48+
Map<String, Object> map = new HashMap();
49+
map.put("name", scenarioOutline.getName());
50+
map.put("description", scenarioOutline.getDescription());
51+
map.put("line", scenarioOutline.getLine());
52+
map.put("sectionIndex", scenarioOutline.getSection().getIndex());
53+
map.put("exampleTableCount", scenarioOutline.getNumExampleTables());
54+
map.put("exampleTables", scenarioOutline.getAllExampleData());
55+
map.put("numScenariosToExecute", scenarioOutline.getNumScenarios());
56+
57+
// Get results of other examples in this outline
58+
List<Map<String, Object>> scenarioResults = new ArrayList();
59+
if (runtime.featureRuntime != null && runtime.featureRuntime.result != null) {
60+
// Add all past results
61+
boolean needToAddRecent = runtime.result != null;
62+
for(ScenarioResult result : runtime.featureRuntime.result.getScenarioResults()) {
63+
if (result.getScenario().getSection().getIndex() == scenarioOutline.getSection().getIndex()) {
64+
scenarioResults.add(result.toInfoJson());
65+
if(result.equals(runtime.result)) {
66+
needToAddRecent = false;
67+
}
68+
}
69+
}
70+
71+
// Add most recent result if we haven't already (and it's not null)
72+
if (needToAddRecent) {
73+
scenarioResults.add(runtime.result.toInfoJson());
74+
}
75+
}
76+
map.put("scenarioResults", scenarioResults);
77+
map.put("numScenariosExecuted", scenarioResults.size());
78+
79+
return map;
80+
}
81+
82+
}

0 commit comments

Comments
 (0)