Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tank API V2 Clients Release #245

Merged
merged 73 commits into from
Jun 29, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
73 commits
Select commit Hold shift + click to select a range
cc89064
feat: Add V2 clients
Mar 23, 2023
0f5989d
fix: clean up + switching to webclient exception handling
Mar 23, 2023
efe65c6
fix: better exception handling
Mar 24, 2023
66d2ca4
feat: [SRE-26521] upload/download for scripts/datafiles
Mar 20, 2023
2fe9b7e
Revert "feat: [SRE-26521] upload/download for scripts/datafiles"
Mar 27, 2023
630a5cb
Merge branch 'master' into zkofiro/v2-clients
Mar 27, 2023
96c58f8
feat: Updated clients, new routes, status response from agent/job ops…
Mar 28, 2023
c550a9e
fix: listing only user-facing endpoints
Mar 28, 2023
ae9923d
fix: small updates to deps + docs for client changes
Mar 28, 2023
2e70ebb
fix: temp fix for tools while testing agent
Mar 28, 2023
5c96250
fix: removing unneeded files
Mar 28, 2023
567757d
Merge branch 'master' into zkofiro/v2-clients
Apr 7, 2023
bb9e9eb
fix: temp fix for client tools
Apr 7, 2023
24c8ba5
fix: missing file
Apr 7, 2023
b84720f
moved last backend V1 deps over with models + tested locally
Apr 13, 2023
b2a0e7a
updating file documentation
Apr 13, 2023
5167739
reverting debug flag
Apr 13, 2023
d98d611
moving other misc backend changes to this branch
Apr 24, 2023
6259919
moving over one more backend change
Apr 24, 2023
7ac6a61
updated in UI ref branch
May 3, 2023
a096470
Merge branch 'zkofiro/move-remaining-V1' into zkofiro/SRE-26707
May 3, 2023
42d5651
Merge branch 'master' into zkofiro/v2-clients
Jun 5, 2023
99c5754
Merge branch 'master' into zkofiro/SRE-26707
Jun 5, 2023
e29c166
Merge branch 'zkofiro/SRE-26707' into zkofiro/v2-clients
Jun 6, 2023
997ebde
cleaning up, removing old files
Jun 6, 2023
cf8a544
testing base agent client
Jun 6, 2023
875e8f6
small clean up
Jun 6, 2023
9287885
dep injection fix
Jun 7, 2023
59ba285
need to shorten V2 url due to dep inj fix
Jun 7, 2023
f587553
missing dependency
Jun 8, 2023
b3e268d
unneeded files
Jun 8, 2023
e072422
reverting cloud models until move away from Java EE CDI
Jun 8, 2023
6cdbd45
updating vmtracker as well
Jun 8, 2023
c3d5b4d
TankAPI fix
Jun 8, 2023
c0531a0
moving vmmanger + models outside of rest api
Jun 9, 2023
101e509
update pom for logging
Jun 9, 2023
abb65a0
vmTracker models moved to api
Jun 9, 2023
b44a5c5
updating import ref to cloud
Jun 9, 2023
c1247bc
reverting defaultcontroller
Jun 9, 2023
816d1c5
revert ui for now
Jun 9, 2023
3a7b524
reverting module deletion, saving for V1 removal
Jun 9, 2023
834f218
reverting local files
Jun 9, 2023
d089dce
updating package names
Jun 9, 2023
ea8d78a
update UI refs to /v2 path
Jun 9, 2023
ee46bcc
banner fixed to stay closed each session
Jun 11, 2023
10bc49e
updated tools for changes
Jun 12, 2023
10ff8ab
banner logic for SSOlogin
Jun 12, 2023
ec0bae4
exclude to reenable agent logging
Jun 12, 2023
66b281b
commenting back httpclient tests
Jun 12, 2023
69b9c5b
Merge branch 'master' into zkofiro/v2-clients
Jun 12, 2023
5985ffa
should fix agent logging
Jun 12, 2023
af5f468
reverting files
Jun 12, 2023
5173403
adding to child pom
Jun 12, 2023
a12061e
logback vs log4j2 mismatch - probable cause for log drop between modules
Jun 13, 2023
5932a36
attempt to force spring to use log4j
Jun 13, 2023
f63c882
correct file path for log4j2
Jun 13, 2023
7f031e2
revert after testing logging fix
Jun 13, 2023
d2ae166
legacy spring startup, removing from web/WEB-INF
Jun 13, 2023
a57438c
re-adding bridge deps
Jun 13, 2023
37ff905
fixed versions
Jun 13, 2023
3778440
logging config issue fix: including maven-shade-plugin fixes issue lo…
Jun 13, 2023
094ef10
updating gitignore
Jun 13, 2023
30d448b
logs showing locally, just needed to exclude extra servlet fragment
Jun 14, 2023
95f4295
switching scope out for absolute-ordering, logging to file locally
Jun 14, 2023
99c4239
moving from tank-api to agent module
Jun 14, 2023
e358551
exclude Log4j2plugins.dat w/ maven shaded plugin instead, avoiding de…
Jun 14, 2023
72bb839
updating fix for other modules, tested locally
Jun 14, 2023
5b9ec82
uncommenting tests now that builds passing
Jun 14, 2023
932b170
updated not needed, pending deletion
Jun 15, 2023
6db2e74
filter endpoint fix - moving script models out to api module
Jun 28, 2023
f627fec
Revert "filter endpoint fix - moving script models out to api module"
Jun 28, 2023
0a2cc49
new filter fix: reverting all script models to v1 until external scri…
Jun 28, 2023
5779312
reduced war file size by 80M, still needs more dep exclusions from sp…
Jun 29, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 39 additions & 1 deletion agent/agent_standalone/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,46 @@
<dependencies>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>agent-client</artifactId>
<artifactId>tank-api</artifactId>
<version>${project.version}</version>
<exclusions>
<exclusion>
<groupId>org.springdoc</groupId>
<artifactId>springdoc-openapi-ui</artifactId>
</exclusion>
<exclusion>
<groupId>${project.groupId}</groupId>
<artifactId>script-engine</artifactId>
</exclusion>
<exclusion>
<groupId>${project.groupId}</groupId>
<artifactId>data-access</artifactId>
</exclusion>
<exclusion>
<groupId>${project.groupId}</groupId>
<artifactId>tank-vm-manager</artifactId>
</exclusion>
<exclusion>
<groupId>${project.groupId}</groupId>
<artifactId>tank-script-processor</artifactId>
</exclusion>
<exclusion>
<groupId>${project.groupId}</groupId>
<artifactId>api</artifactId>
</exclusion>
<exclusion>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
</exclusion>
<exclusion>
<groupId>jakarta.xml.bind</groupId>
<artifactId>jakarta.xml.bind-api</artifactId>
</exclusion>
</exclusions>
</dependency>

<dependency>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;

import com.intuit.tank.AgentServiceClient;
import com.intuit.tank.rest.mvc.rest.clients.AgentClient;
import com.intuit.tank.harness.HostInfo;
import com.intuit.tank.vm.agent.messages.AgentAvailability;
import com.intuit.tank.vm.agent.messages.AgentAvailabilityStatus;
Expand All @@ -37,23 +37,23 @@
public class StandaloneAgentStartup implements Runnable {

private static Logger LOG = LogManager.getLogger(StandaloneAgentStartup.class);
public static final String SERVICE_RELATIVE_PATH = "/rest/v1/agent-service";
public static final String SERVICE_RELATIVE_PATH = "/v2/agent";
private static String API_HARNESS_COMMAND = "./startAgent.sh";
public static final String METHOD_SETTINGS = "/settings";
public static final String METHOD_SUPPORT = "/supportFiles";
public static final String METHOD_SUPPORT = "/support-files";
private static final long PING_TIME = 1000 * 60 * 5;// five minutes

private String controllerBase;
private AgentAvailability currentAvailability;
private AgentServiceClient agentClient;
private AgentClient agentClient;
private String instanceId;
private String hostname;
private int capacity = 4000;

@Override
public void run() {
CommandListener.startHttpServer(CommandListener.PORT, this);
agentClient = new AgentServiceClient(controllerBase);
agentClient = new AgentClient(controllerBase);

if (hostname != null) {
instanceId = hostname;
Expand Down Expand Up @@ -152,7 +152,7 @@ private void sendAvailability() {
currentAvailability.getInstanceUrl(), currentAvailability.getCapacity(),
currentAvailability.getAvailabilityStatus());
LOG.info("Sending availaability: " + ToStringBuilder.reflectionToString(availability));
agentClient.standaloneAgentAvailable(availability);
agentClient.setStandaloneAgentAvailability(availability);
}

public static void main(String[] args) {
Expand Down
5 changes: 5 additions & 0 deletions agent/agent_standalone_pkg/src/main/assembly/assembly.xml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,11 @@
<useProjectArtifact>true</useProjectArtifact>
<unpack>true</unpack>
<scope>runtime</scope>
<unpackOptions>
<excludes>
<exclude>**/Log4j2Plugins.dat</exclude>
</excludes>
</unpackOptions>
</dependencySet>
</dependencySets>
</assembly>
Original file line number Diff line number Diff line change
Expand Up @@ -34,10 +34,10 @@

public class AgentStartup implements Runnable {
private static final Logger logger = LogManager.getLogger(AgentStartup.class);
private static final String SERVICE_RELATIVE_PATH = "/rest/v1/agent-service";
private static final String SERVICE_RELATIVE_PATH = "/v2/agent";
private static final String METHOD_SETTINGS = "/settings";
private static final String API_HARNESS_COMMAND = "./startAgent.sh";
private static final String METHOD_SUPPORT = "/supportFiles";
private static final String METHOD_SUPPORT = "/support-files";
private static final int[] FIBONACCI = new int[] { 1, 1, 2, 3, 5, 8, 13 };

private final String controllerBaseUrl;
Expand Down Expand Up @@ -120,5 +120,8 @@ private static void usage() {
System.out.println("Tank Test Startup Usage:");
System.out.println("java -cp agent-startup-pkg-1.0-all.jar com/intuit/tank/agent/AgentStartup <options>");
System.out.println("-controller=<controller_base_url>: The url of the controller to get test info from");
System.out.println("Service Path: " + SERVICE_RELATIVE_PATH);
System.out.println("Settings Method: " + METHOD_SETTINGS);
System.out.println("Support Files Method: " + METHOD_SUPPORT);
}
}
45 changes: 44 additions & 1 deletion agent/apiharness/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,50 @@
<version>${project.version}</version>
</dependency>

<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>tank-api</artifactId>
<version>${project.version}</version>
<exclusions>
<exclusion>
<groupId>org.springdoc</groupId>
<artifactId>springdoc-openapi-ui</artifactId>
</exclusion>
<exclusion>
<groupId>${project.groupId}</groupId>
<artifactId>script-engine</artifactId>
</exclusion>
<exclusion>
<groupId>${project.groupId}</groupId>
<artifactId>data-access</artifactId>
</exclusion>
<exclusion>
<groupId>${project.groupId}</groupId>
<artifactId>tank-vm-manager</artifactId>
</exclusion>
<exclusion>
<groupId>${project.groupId}</groupId>
<artifactId>tank-script-processor</artifactId>
</exclusion>
<exclusion>
<groupId>${project.groupId}</groupId>
<artifactId>api</artifactId>
</exclusion>
<exclusion>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
</exclusion>
<exclusion>
<groupId>jakarta.xml.bind</groupId>
<artifactId>jakarta.xml.bind-api</artifactId>
</exclusion>
</exclusions>
</dependency>

<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>agent-client</artifactId>
Expand Down Expand Up @@ -81,5 +125,4 @@
<version>${project.version}</version>
</dependency>
</dependencies>

</project>
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,10 @@
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;

import com.intuit.tank.CloudServiceClient;
import com.intuit.tank.api.model.v1.cloud.CloudVmStatus;
import com.intuit.tank.api.model.v1.cloud.VMStatus;
import com.intuit.tank.api.model.v1.cloud.ValidationStatus;
import com.intuit.tank.rest.mvc.rest.clients.AgentClient;
import com.intuit.tank.vm.vmManager.models.CloudVmStatus;
import com.intuit.tank.vm.vmManager.models.VMStatus;
import com.intuit.tank.vm.vmManager.models.ValidationStatus;
import com.intuit.tank.harness.logging.LogUtil;
import com.intuit.tank.reporting.api.TPSInfoContainer;
import com.intuit.tank.vm.agent.messages.WatsAgentStatusResponse;
Expand All @@ -36,7 +36,7 @@ public class APIMonitor implements Runnable {
private static final int MIN_REPORT_TIME = 15000;
private static Logger LOG = LogManager.getLogger(APIMonitor.class);
private static boolean doMonitor = true;
private static CloudServiceClient client;
private static AgentClient client;
private static CloudVmStatus status;
private long reportInterval = APIMonitor.MIN_REPORT_TIME;
private boolean isLocal;
Expand All @@ -45,7 +45,7 @@ public APIMonitor(Boolean isLocal, CloudVmStatus vmStatus) {
this.isLocal = isLocal;
status = vmStatus;
try {
client = new CloudServiceClient(APITestHarness.getInstance().getTankConfig().getControllerBase());
client = new AgentClient(APITestHarness.getInstance().getTankConfig().getControllerBase());
reportInterval = Math.max(APITestHarness.getInstance().getTankConfig().getAgentConfig()
.getStatusReportIntervalMilis(reportInterval), MIN_REPORT_TIME);
} catch (Exception e) {
Expand All @@ -65,15 +65,15 @@ public void run() {
newStatus.setTotalTps(tpsInfo.getTotalTps());
sendTps(tpsInfo);
}
if (!isLocal) client.setVmStatus(newStatus.getInstanceId(), newStatus);
if (!isLocal) client.setInstanceStatus(newStatus.getInstanceId(), newStatus);
APITestHarness.getInstance().checkAgentThreads();
Thread.sleep(reportInterval);
} catch (Exception t) {
LOG.error(LogUtil.getLogMessage("Unable to send status metrics | " + t.getMessage()), t);
}
}
CloudVmStatus newStatus = createStatus(APITestHarness.getInstance().getStatus());
client.setVmStatus(newStatus.getInstanceId(), newStatus);
client.setInstanceStatus(newStatus.getInstanceId(), newStatus);
}

private void sendTps(final TPSInfoContainer tpsInfo) {
Expand Down Expand Up @@ -136,7 +136,7 @@ public synchronized static void setJobStatus(JobStatus jobStatus) {
stats.getMaxVirtualUsers(),
stats.getCurrentNumberUsers(), status.getStartTime(), endTime);
status.setUserDetails(APITestHarness.getInstance().getUserTracker().getSnapshot());
client.setVmStatus(status.getInstanceId(), status);
client.setInstanceStatus(status.getInstanceId(), status);
} catch (Exception e) {
LOG.error("Error sending status to controller: " + e.toString(), e);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,10 +41,10 @@
import org.apache.logging.log4j.core.config.LoggerConfig;
import org.apache.logging.log4j.message.ObjectMessage;

import com.intuit.tank.AgentServiceClient;
import com.intuit.tank.api.model.v1.cloud.CloudVmStatus;
import com.intuit.tank.api.model.v1.cloud.VMStatus;
import com.intuit.tank.api.model.v1.cloud.ValidationStatus;
import com.intuit.tank.rest.mvc.rest.clients.AgentClient;
import com.intuit.tank.vm.vmManager.models.CloudVmStatus;
import com.intuit.tank.vm.vmManager.models.VMStatus;
import com.intuit.tank.vm.vmManager.models.ValidationStatus;
import com.intuit.tank.harness.data.HDTestPlan;
import com.intuit.tank.harness.data.HDWorkload;
import com.intuit.tank.harness.logging.LogUtil;
Expand Down Expand Up @@ -252,7 +252,7 @@ private void startHttp(String baseUrl) {
if (baseUrl == null) {
baseUrl = AmazonUtil.getControllerBaseUrl();
}
AgentServiceClient client = new AgentServiceClient(baseUrl);
AgentClient client = new AgentClient(baseUrl);
String instanceUrl = null;
int retryCount = 0;
while (instanceUrl == null) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
import java.util.Map;
import java.util.stream.Collectors;

import com.intuit.tank.api.model.v1.cloud.UserDetail;
import com.intuit.tank.vm.vmManager.models.UserDetail;

/**
*
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
package com.intuit.tank.harness;

import com.intuit.tank.api.model.v1.cloud.CloudVmStatus;
import com.intuit.tank.api.model.v1.cloud.VMStatus;
import com.intuit.tank.api.model.v1.cloud.ValidationStatus;
import com.intuit.tank.vm.vmManager.models.CloudVmStatus;
import com.intuit.tank.vm.vmManager.models.VMStatus;
import com.intuit.tank.vm.vmManager.models.ValidationStatus;
import com.intuit.tank.vm.api.enumerated.JobStatus;
import com.intuit.tank.vm.api.enumerated.VMImageType;
import com.intuit.tank.vm.api.enumerated.VMRegion;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@

import java.util.List;

import com.intuit.tank.api.model.v1.cloud.UserDetail;
import com.intuit.tank.vm.vmManager.models.UserDetail;
import com.intuit.tank.test.TestGroups;
import org.junit.jupiter.api.Tag;
import org.junit.jupiter.api.Test;
Expand Down
5 changes: 5 additions & 0 deletions agent/apiharness_pkg/src/main/assembly/assembly.xml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,11 @@
<useProjectArtifact>true</useProjectArtifact>
<unpack>true</unpack>
<scope>runtime</scope>
<unpackOptions>
<excludes>
<exclude>**/Log4j2Plugins.dat</exclude>
</excludes>
</unpackOptions>
</dependencySet>
</dependencySets>
</assembly>
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ private ProjectServiceUrlBuilder() {
*/
public static String getScriptXmlUrl(String jobId) {
String baseUrl = new TankConfig().getControllerBase();
return baseUrl + "/rest/v1/project-service/script/" + jobId;
return baseUrl + "/v2/jobs/script/" + jobId;
}

}
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
/**
* Copyright 2011 Intuit Inc. All Rights Reserved
*/
package com.intuit.tank.api.cloud;
package com.intuit.tank.vm.vmManager;

/*
* #%L
Expand All @@ -20,9 +20,9 @@

import javax.annotation.Nonnull;

import com.intuit.tank.api.model.v1.cloud.CloudVmStatus;
import com.intuit.tank.api.model.v1.cloud.CloudVmStatusContainer;
import com.intuit.tank.api.model.v1.cloud.ProjectStatusContainer;
import com.intuit.tank.vm.vmManager.models.CloudVmStatus;
import com.intuit.tank.vm.vmManager.models.CloudVmStatusContainer;
import com.intuit.tank.vm.vmManager.models.ProjectStatusContainer;
import com.intuit.tank.vm.event.JobEvent;

/**
Expand Down
Loading