java.lang.OutOfMemoryError: Metaspace happen on appium java client 9.2.3 #2247

luckhiem · 2024-12-02T09:35:16Z

Description

I upgraded the framework from v7 to java-client v9, but after upgrade, we got java.lang.OutOfMemoryError: Metaspace, the test scripts is the same and we not facing this problem in v7 and java8

Environment

Java client build version or git revision if you use some snapshot: 9.2.3
Appium server version or git revision if you use some snapshot: appium v2
Desktop OS/version used to run Appium if necessary: MacOS
Node.js version (unless using Appium.app|exe) or Appium CLI or Appium.app|exe: v23.2.0
Mobile platform/version under test: Android
Real device or emulator/simulator: real device

Details

 io.cucumber.core.exception.CucumberException: Failed to instantiate class stepDefinitions.InitialSteps
Caused by: java.lang.OutOfMemoryError: Metaspace

Code To Reproduce Issue [ Good To Have ]

pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>groupId</groupId>
    <artifactId>7eleven-automate-test</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>jar</packaging>

    <properties>
        <maven.compiler.source>11</maven.compiler.source>
        <maven.compiler.target>11</maven.compiler.target>
        <testng.version>7.3.0</testng.version>
        <appium.version>9.2.3</appium.version>
        <hamcrest.version>1.3</hamcrest.version>
        <cucumber.version>7.18.1</cucumber.version>
        <extent.report.version>2.41.2</extent.report.version>
        <cucumber.reporting.version>5.3.1</cucumber.reporting.version>
        <slf4j.version>2.0.13</slf4j.version>
    </properties>

    <dependencies>
        <!-- Cucumber Dependencies -->
        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-core</artifactId>
            <version>${cucumber.version}</version>
        </dependency>
        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-java</artifactId>
            <version>${cucumber.version}</version>
        </dependency>
        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-testng</artifactId>
            <version>${cucumber.version}</version>
            <scope>test</scope>
            <exclusions>
                <exclusion>
                    <groupId>junit</groupId>
                    <artifactId>junit</artifactId>
                </exclusion>
            </exclusions>
        </dependency>

        <!-- Appium and Selenium Dependencies -->
        <dependency>
            <groupId>io.appium</groupId>
            <artifactId>java-client</artifactId>
            <version>${appium.version}</version>
            <scope>test</scope>
        </dependency>

        <!-- TestNG -->
        <dependency>
            <groupId>org.testng</groupId>
            <artifactId>testng</artifactId>
            <version>${testng.version}</version>
            <scope>test</scope>
        </dependency>

        <!-- Other Dependencies -->
        <dependency>
            <groupId>org.hamcrest</groupId>
            <artifactId>hamcrest-all</artifactId>
            <version>${hamcrest.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>net.masterthought</groupId>
            <artifactId>cucumber-reporting</artifactId>
            <version>${cucumber.reporting.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>com.relevantcodes</groupId>
            <artifactId>extentreports</artifactId>
            <version>${extent.report.version}</version>
        </dependency>
        <dependency>
            <groupId>com.squareup.okhttp3</groupId>
            <artifactId>okhttp</artifactId>
            <version>4.9.3</version>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-api</artifactId>
            <version>${slf4j.version}</version>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-simple</artifactId>
            <version>${slf4j.version}</version>
        </dependency>
        <dependency>
            <groupId>com.googlecode.json-simple</groupId>
            <artifactId>json-simple</artifactId>
            <version>1.1.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-lang3</artifactId>
            <version>3.15.0</version>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <version>1.18.34</version>
            <scope>provided</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <!-- Compiler Plugin -->
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.10.1</version>
                <configuration>
                    <source>${maven.compiler.source}</source>
                    <target>${maven.compiler.target}</target>
                </configuration>
            </plugin>

            <!-- Surefire Plugin -->
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>3.0.0-M9</version>
                <configuration>
                    <suiteXmlFiles>
                        <suiteXmlFile>${project.basedir}/src/test/resources/TestSuite/testng.xml</suiteXmlFile>
                        <argLine>-XX:MetaspaceSize=256m -XX:MaxMetaspaceSize=512m</argLine>
                    </suiteXmlFiles>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

The text was updated successfully, but these errors were encountered:

valfirst · 2024-12-02T10:24:21Z

@luckhiem please check #2119

luckhiem · 2024-12-02T11:18:23Z

@valfirst I'm using Appium v9.2.3 so it should be fixed right?

luckhiem · 2024-12-02T11:27:29Z

@mykola-mokhnach Do you have any idea on that?

mykola-mokhnach · 2024-12-02T11:35:20Z

I would try to increase the maximum metaspace size in JVM params: https://www.mastertheboss.com/java/solving-java-lang-outofmemoryerror-metaspace-error/

valfirst · 2024-12-02T12:40:32Z

@valfirst I'm using Appium v9.2.3 so it should be fixed right?

the issue I provided may give you a clue why it happens, in general it would be nice to have a sample project to reproduce the issue

luckhiem · 2024-12-02T13:39:58Z

@mykola-mokhnach already increase the maximum metaspace size in JVM params but no luck still failing with same issue, I just run test in 5mins but the metaspace increase look abnormal

mykola-mokhnach · 2024-12-02T17:37:18Z

Like @valfirst mentioned we would need a sample project then to be able to reproduce the issue locally

luckhiem · 2024-12-03T06:21:36Z

to be honest, this isa private project so I can not share it, but I'm doing something really simple like

BasePage

package pages;

import io.appium.java_client.AppiumDriver;
import io.appium.java_client.pagefactory.AppiumFieldDecorator;
import org.openqa.selenium.support.PageFactory;
import utils.DriverManager;

public class BasePage {
    protected AppiumDriver driver;

    public BasePage() {
        this.driver = DriverManager.getDriver();
        PageFactory.initElements(new AppiumFieldDecorator(driver), this);
    }

    public void clickElement(MobileElement element) {
        element.click();
    }

    public void enterText(MobileElement element, String text) {
        element.sendKeys(text);
    }
}

LoginPage

package pages;

import io.appium.java_client.MobileElement;
import io.appium.java_client.pagefactory.AndroidFindBy;

public class LoginPage extends BasePage {
    @AndroidFindBy(id = "com.example.app:id/username")
    private MobileElement usernameField;

    @AndroidFindBy(id = "com.example.app:id/password")
    private MobileElement passwordField;

    @AndroidFindBy(id = "com.example.app:id/login")
    private MobileElement loginButton;

    public void enterUsername(String username) {
        enterText(usernameField, username);
    }

    public void enterPassword(String password) {
        enterText(passwordField, password);
    }

    public void clickLogin() {
        clickElement(loginButton);
    }
}

package stepdefinitions;

import io.cucumber.java.en.Given;
import io.cucumber.java.en.Then;
import io.cucumber.java.en.When;
import pages.LoginPage;
import utils.DriverManager;

public class LoginSteps {
    private final LoginPage loginPage = new LoginPage();

    @Given("the app is launched")
    public void theAppIsLaunched() {
        DriverManager.initializeDriver();
    }

    @When("I enter a valid username and password")
    public void enterValidCredentials() {
        loginPage.enterUsername("testuser");
        loginPage.enterPassword("password123");
    }

    @When("I click the login button")
    public void clickLoginButton() {
        loginPage.clickLogin();
    }

    @Then("I should see the home screen")
    public void verifyHomeScreen() {
        // Add assertion logic here to validate the home screen.
    }
}

LoginFeature

Feature: Login Feature
  Scenario: Valid Login
    Given the app is launched
    When I enter a valid username and password
    And I click the login button
    Then I should see the home screen

pom

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>groupId</groupId>
    <artifactId>7eleven-automate-test</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>jar</packaging>

    <properties>
        <maven.compiler.source>11</maven.compiler.source>
        <maven.compiler.target>11</maven.compiler.target>
        <testng.version>7.3.0</testng.version>
        <appium.version>9.2.3</appium.version>
        <hamcrest.version>1.3</hamcrest.version>
        <cucumber.version>7.18.1</cucumber.version>
        <extent.report.version>2.41.2</extent.report.version>
        <cucumber.reporting.version>5.3.1</cucumber.reporting.version>
        <slf4j.version>2.0.13</slf4j.version>
    </properties>

    <dependencies>
        <!-- Cucumber Dependencies -->
        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-core</artifactId>
            <version>${cucumber.version}</version>
        </dependency>
        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-java</artifactId>
            <version>${cucumber.version}</version>
        </dependency>
        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-testng</artifactId>
            <version>${cucumber.version}</version>
            <scope>test</scope>
            <exclusions>
                <exclusion>
                    <groupId>junit</groupId>
                    <artifactId>junit</artifactId>
                </exclusion>
            </exclusions>
        </dependency>

        <!-- Appium and Selenium Dependencies -->
        <dependency>
            <groupId>io.appium</groupId>
            <artifactId>java-client</artifactId>
            <version>${appium.version}</version>
            <scope>test</scope>
        </dependency>

        <!-- TestNG -->
        <dependency>
            <groupId>org.testng</groupId>
            <artifactId>testng</artifactId>
            <version>${testng.version}</version>
            <scope>test</scope>
        </dependency>

        <!-- Other Dependencies -->
        <dependency>
            <groupId>org.hamcrest</groupId>
            <artifactId>hamcrest-all</artifactId>
            <version>${hamcrest.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>net.masterthought</groupId>
            <artifactId>cucumber-reporting</artifactId>
            <version>${cucumber.reporting.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>com.relevantcodes</groupId>
            <artifactId>extentreports</artifactId>
            <version>${extent.report.version}</version>
        </dependency>
        <dependency>
            <groupId>com.squareup.okhttp3</groupId>
            <artifactId>okhttp</artifactId>
            <version>4.9.3</version>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-api</artifactId>
            <version>${slf4j.version}</version>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-simple</artifactId>
            <version>${slf4j.version}</version>
        </dependency>
        <dependency>
            <groupId>com.googlecode.json-simple</groupId>
            <artifactId>json-simple</artifactId>
            <version>1.1.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-lang3</artifactId>
            <version>3.15.0</version>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <version>1.18.34</version>
            <scope>provided</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <!-- Compiler Plugin -->
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.10.1</version>
                <configuration>
                    <source>${maven.compiler.source}</source>
                    <target>${maven.compiler.target}</target>
                </configuration>
            </plugin>

            <!-- Surefire Plugin -->
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>3.0.0-M9</version>
                <configuration>
                    <suiteXmlFiles>
                        <suiteXmlFile>${project.basedir}/src/test/resources/TestSuite/testng.xml</suiteXmlFile>
                        <argLine>-XX:MetaspaceSize=256m -XX:MaxMetaspaceSize=512m</argLine>
                    </suiteXmlFiles>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

After trying my project with Appium 8.5.0 and Selenium 4.13.0 the performance look normal now:

valfirst · 2024-12-03T07:11:18Z

@luckhiem would it be possible to provide a sample without Cucumber? from your sample it's not clear how to inject steps into the configuration

ideally it would be nice to have a github project with one line command to run and reproduce the problem

RitikaVashist2890 · 2024-12-06T15:43:27Z

Hi @luckhiem, I encountered a similar issue in our project. In our case, an excessive number of calls to Byte Buddy were occurring, and the class loader count was too high. To address this, we added a Byte Buddy dependency in the POM.xml to prevent random proxy calls. Additionally, we reviewed all dependencies, removed redundancies, and upgraded to Java 17 and the latest java-client version (9.3). These changes resolved the issue for us. Might be you can also also analyze your classes on runtime and check what exactly is causing issue.

We also configured the following Maven Surefire Plugin to optimize test execution:

    <plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.5.2</version>
<configuration>
    <testFailureIgnore>true</testFailureIgnore>
    <forkCount>3</forkCount>
    <forkedProcessExitTimeoutInSeconds>1200</forkedProcessExitTimeoutInSeconds>
    <argLine>-XX:+UseG1GC -XX:MaxMetaspaceSize=8g -XX:CompressedClassSpaceSize=3g -XX:+ClassUnloadingWithConcurrentMark -Xms250m -Xmx8g</argLine>
</configuration>
</plugin>

I hope it will help you .

Auto81 · 2025-01-08T11:22:16Z

Seeing similar strangeness, I have been able to reproduce a simple example here https://github.com/Auto81/SeleniumMemoryLeak

I can monitor using VisualVM as the memory usage continues to grow through out the test run.

Please let me know if I can provide anything else or need to move this to separate issue, thank you :)

Auto81 · 2025-01-14T14:56:51Z

@mykola-mokhnach @valfirst Sorry to tag yous directly

This looks like it's the AppiumFieldDecorator within PageFactory.initElements(new AppiumFieldDecorator(driver), this); that is the entry point for triggering this memory leak, would yous agree?

My attached sample project has it laid out for versions of selenium it does and does not work with.

https://github.com/Auto81/SeleniumMemoryLeak

mykola-mokhnach · 2025-01-14T15:08:14Z

Unfortunately I don't have much time to investigate this issue. If someone wants to step in then feel free to do that.

asolntsev · 2025-01-14T23:12:43Z

As far as I see, the cause of memory leak is static field io.appium.java_client.proxy.Helpers#CACHED_PROXY_CLASSES.

With Selenium 4.25.0, it holds only once instance.
With Selenium 4.26.0, it grows indefinitely, holding multiple instances of ProxyClassSignature - this causes OOM.

Apparently, caused by https://github.com/appium/java-client/pull/2129/files @mykola-mokhnach

amedvedjev · 2025-01-30T13:10:29Z

java-client 9.3.0, selenium 4.23 - no issue
java-client 9.3.0, selenium 4.25 - no issue
java-client 9.3.0, selenium 4.26 - issue reproducing
java-client 9.4.0, selenium 4.26 - issue reproducing
java-client 9.4.0, selenium 4.28 - issue reproducing

@valfirst @mykola-mokhnach

same simple code helps to reproduce issue -> #2119 (comment)

Auto81 · 2025-01-30T13:28:48Z

java-client 9.3.0, selenium 4.23 - no issue java-client 9.3.0, selenium 4.25 - no issue java-client 9.3.0, selenium 4.26 - issue reproducing java-client 9.4.0, selenium 4.26 - issue reproducing java-client 9.4.0, selenium 4.28 - issue reproducing

@valfirst @mykola-mokhnach

same simple code helps to reproduce issue -> #2119 (comment)

Have you tried with the jvm args highlight above? #2247 (comment)

amedvedjev · 2025-01-30T14:35:39Z

added

<dependency>
    <groupId>net.bytebuddy</groupId>
    <artifactId>byte-buddy</artifactId>
    <version>1.16.1</version>
</dependency>

->
java-client 9.3.0, selenium 4.26 - no issue
java-client 9.4.0, selenium 4.28 - no issue

wow! thanks! just missed that comment. BUT I still believe issues should be fixed on java-client side also.

Auto81 · 2025-01-30T14:42:43Z

@amedvedjev totally agreed but untill someone has time to properly resolve, it's the best we have

Auto81 mentioned this issue Jan 8, 2025

[🐛 Bug]: Selenium 4.26.0 has memory leak. SeleniumHQ/selenium#14789

Closed

Auto81 mentioned this issue Jan 20, 2025

fix: Use WeakHashMap for caching proxy classes #2260

Merged

4 tasks

valfirst linked a pull request Jan 20, 2025 that will close this issue

fix: Use WeakHashMap for caching proxy classes #2260

Merged

4 tasks

mykola-mokhnach closed this as completed in #2260 Jan 20, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

java.lang.OutOfMemoryError: Metaspace happen on appium java client 9.2.3 #2247

java.lang.OutOfMemoryError: Metaspace happen on appium java client 9.2.3 #2247

luckhiem commented Dec 2, 2024 •

edited

Loading

valfirst commented Dec 2, 2024

luckhiem commented Dec 2, 2024

luckhiem commented Dec 2, 2024

mykola-mokhnach commented Dec 2, 2024

valfirst commented Dec 2, 2024

luckhiem commented Dec 2, 2024

mykola-mokhnach commented Dec 2, 2024

luckhiem commented Dec 3, 2024

valfirst commented Dec 3, 2024

RitikaVashist2890 commented Dec 6, 2024 •

edited

Loading

Auto81 commented Jan 8, 2025

Auto81 commented Jan 14, 2025

mykola-mokhnach commented Jan 14, 2025

asolntsev commented Jan 14, 2025

amedvedjev commented Jan 30, 2025

Auto81 commented Jan 30, 2025

amedvedjev commented Jan 30, 2025

Auto81 commented Jan 30, 2025

java.lang.OutOfMemoryError: Metaspace happen on appium java client 9.2.3 #2247

java.lang.OutOfMemoryError: Metaspace happen on appium java client 9.2.3 #2247

Comments

luckhiem commented Dec 2, 2024 • edited Loading

Description

Environment

Details

Code To Reproduce Issue [ Good To Have ]

valfirst commented Dec 2, 2024

luckhiem commented Dec 2, 2024

luckhiem commented Dec 2, 2024

mykola-mokhnach commented Dec 2, 2024

valfirst commented Dec 2, 2024

luckhiem commented Dec 2, 2024

mykola-mokhnach commented Dec 2, 2024

luckhiem commented Dec 3, 2024

valfirst commented Dec 3, 2024

RitikaVashist2890 commented Dec 6, 2024 • edited Loading

Auto81 commented Jan 8, 2025

Auto81 commented Jan 14, 2025

mykola-mokhnach commented Jan 14, 2025

asolntsev commented Jan 14, 2025

amedvedjev commented Jan 30, 2025

Auto81 commented Jan 30, 2025

amedvedjev commented Jan 30, 2025

Auto81 commented Jan 30, 2025

luckhiem commented Dec 2, 2024 •

edited

Loading

RitikaVashist2890 commented Dec 6, 2024 •

edited

Loading