performance issue with Page::getCurentUrl()

[divinity76]

I recently have to write shit like

        $currentUrl = $page->evaluate('document.location.href')->getReturnValue();
        // don't use getCurrentUrl() because it has a performance issue... don't have time to investigate now

because when I write

        $t = microtime(true);
        $currentUrl = $page->getCurrentUrl();
        $t = microtime(true) - $t;
        echo "\ngetCurrentUrl() took {$t} seconds." . PHP_EOL;

i sometimes often get crazy readings like

getCurrentUrl() took 0.46461796760559 seconds.
getCurrentUrl() took 2.9382970333099 seconds.
getCurrentUrl() took 4.4821469783783 seconds.

but when I write

        $t = microtime(true);
        //$currentUrl = $page->getCurrentUrl();
        // don't use getCurrentUrl() because it has a performance issue... don't have time to investigate now
        $currentUrl = $page->evaluate('document.location.href')->getReturnValue();
        $t = microtime(true) - $t;
        echo "\ngetCurrentUrl() took {$t} seconds." . PHP_EOL;

I consistently get fast readings like

getCurrentUrl() took 0.00039219856262207 seconds.
getCurrentUrl() took 0.00027704238891602 seconds.
getCurrentUrl() took 0.00028014183044434 seconds.
getCurrentUrl() took 0.00026392936706543 seconds.
getCurrentUrl() took 0.00025701522827148 seconds.

WSL Ubuntu 24.04 x86-64
chrome-php/chrome 1.14.0
chrome-php/wrench 1.7.1
php 8.4.12
Chromium 139.0.7258.138 snap

[divinity76]

Unable to share the actual script where it happens.

Tried making a simple reproduce script, but failed. This code does not reproduce the issue.

<?php

declare(strict_types=1);
require_once(__DIR__ . '/../composer/vendor/autoload.php');
$chromeBinary = "/snap/bin/chromium";
$browser_factory = new \HeadlessChromium\BrowserFactory($chromeBinary);
$browser_factory->setOptions([
    "headless" => true,
    "noSandbox" => true,
    'customFlags' => [
        // docker ...
        '--disable-dev-shm-usage',
    ]
]);
$browser = $browser_factory->createBrowser();
$page = $browser->createPage();
$highscore = -INF;
for ($i = 0; $i < 100; ++$i) {
    $page->navigate("http://example.com")->waitForNavigation();
    $t = microtime(true);
    $url = $page->getCurrentUrl();
    $time_taken = microtime(true) - $t;
    if ($time_taken > $highscore) {
        $highscore = $time_taken;
        echo "New highscore: {$highscore} seconds for getCurrentUrl() on {$url}" . PHP_EOL;
    }
    $page->navigate("http://example.org")->waitForNavigation();
    $t = microtime(true);
    $url = $page->getCurrentUrl();
    $time_taken = microtime(true) - $t;
    if ($time_taken > $highscore) {
        $highscore = $time_taken;
        echo "New highscore: {$highscore} seconds for getCurrentUrl() on {$url}" . PHP_EOL;
    }
}

output

New highscore: 0.0035061836242676 seconds for getCurrentUrl() on https://example.com/
New highscore: 0.0035238265991211 seconds for getCurrentUrl() on https://example.com/
New highscore: 0.0040900707244873 seconds for getCurrentUrl() on https://example.org/
New highscore: 0.019552946090698 seconds for getCurrentUrl() on https://example.com/
New highscore: 0.02010703086853 seconds for getCurrentUrl() on https://example.com/
New highscore: 0.022991180419922 seconds for getCurrentUrl() on https://example.com/

[stale]

This issue has been automatically marked as stale because there has been no recent activity. It will be closed after 30 days if no further activity occurs. Thank you for your contributions.

[bsienn-khan]

I just downloaded the latest package and the latest chrome stable release using npx puppeter cli.

This takes 5 seconds on my local mac machine, thanks to @divinity76 his shortcut works best.
So yes, there IS an issue.

$factory = new BrowserFactory(config('app.chrome_binary'));
            $browser = $factory->createBrowser([
                'keepAlive' => true,
                'noSandbox' => true,
                'windowSize' => [1920, 1000],
                'disableNotifications' => true,
                'customFlags' => [
                    '--disable-gpu',                // Essential for some servers/environments
                    '--disable-dev-shm-usage',      // Prevents crashes in Docker/limited memory
                    '--no-sandbox',                 // Bypasses security layer (faster, but use only for trusted content)
                    '--disable-extensions',         // Prevents loading add-ons
                    '--disable-background-networking', // Cuts off noise
                    '--hide-scrollbars',            // Prevents scrollbars from appearing in the PDF
                    '--disable-default-apps',
                    '--disable-sync',
                    '--metrics-recording-only',
                ],
            ]);


        $current_page = array_first($browser->getPages());
        $currentUrl = $current_page->getCurrentUrl();

[divinity76]

@GrahamCampbell @enricodias this issue should be reopened. I don't have permission to reopen it though.

[bsienn-khan]

I just noticed, this block of code is the culprit.
$this->getSession()->getConnection()->readData(); specifically readData()

This is in getCurrentUrl and PageNavigation construct as well.
So my localhost url is also taking 5 seconds to navigate.

This can't be right, something is wrong. Can't use navigate with this slow response.
Makes it useless to work with chrome-php.

Used other php package and it runs under 3 seconds including opening a fresh browser instance.

And they closed the exact issue without response: #646
Awesome work!

[divinity76]

@bsienn-khan Nice find! Caused by inactivity. I can also reproduce it with

<?php

declare(strict_types=1);
require_once(__DIR__ . '/../vendor/autoload.php');
$chromeBinary = "/snap/bin/chromium";
$browser_factory = new \HeadlessChromium\BrowserFactory($chromeBinary);
$browser_factory->setOptions([
    "headless" => true,
    "noSandbox" => true,
    'customFlags' => [
        // docker ...
        '--disable-dev-shm-usage',
    ]
]);
$browser = $browser_factory->createBrowser();
$page = $browser->createPage();
$highscore = -INF;
$page->navigate("http://example.com")->waitForNavigation();
for ($i = 0; $i < 100; ++$i) {
    $t = microtime(true);
    $url = $page->getCurrentUrl();
    $time_taken = microtime(true) - $t;
    if ($time_taken > $highscore) {
        $highscore = $time_taken;
        echo "New highscore: {$highscore} seconds for getCurrentUrl() on {$url}" . PHP_EOL;
    }
}

output

hans@DESKTOP-EE15SLU:~/projects/chrome/hans$ php reproduce.php 
New highscore: 1.0967254638672E-5 seconds for getCurrentUrl() on https://example.com/
New highscore: 0.0005338191986084 seconds for getCurrentUrl() on https://example.com/
New highscore: 0.021073818206787 seconds for getCurrentUrl() on https://example.com/
New highscore: 0.96735405921936 seconds for getCurrentUrl() on https://example.com/
New highscore: 3.1658289432526 seconds for getCurrentUrl() on https://example.com/
New highscore: 5.0055141448975 seconds for getCurrentUrl() on https://example.com/
New highscore: 5.0062091350555 seconds for getCurrentUrl() on https://example.com/
New highscore: 5.0073070526123 seconds for getCurrentUrl() on https://example.com/
New highscore: 5.0073978900909 seconds for getCurrentUrl() on https://example.com/
New highscore: 5.0115480422974 seconds for getCurrentUrl() on https://example.com/

Now I think I know how to fix it! The code should run a socket_select to check if there is data to read, rather than doing an unconditional read!

[bsienn-khan]

@divinity76 Nice, good to know. This was beyond my skillset so couldn't have found the anser you did. Hope you can pr and get it merged soon. Have a good day. Cheers

[divinity76]

I have tracked the actual slow read down to this specific fread() in \Wrench\Socket\AbstractSocket::receive
https://github.com/chrome-php/wrench/blob/bd67a315cf2143ba30598339ad1b6c346db96c99/src/Socket/AbstractSocket.php#L246

that fread is blocking for multiple seconds when this issue occur

[divinity76]

Managed to fix it with stream_select()😄

    private $hasBeenFreadInitialized = false;
    public function receive(int $length = self::DEFAULT_RECEIVE_LENGTH): string
    {
        $buffer = '';
        $metadata['unread_bytes'] = 0;
        $makeBlockingAfterRead = false;
        try {
            do {
                // feof means socket has been closed
                // also, sometimes in long running processes the system seems to kill the underlying socket
                if (!$this->socket || \feof($this->socket)) {
                    $this->disconnect();

                    return $buffer;
                }
                $readArray = [$this->socket];
                $writeArray = null;
                $exceptArray = null;
                $numChangedStreams = \stream_select($readArray, $writeArray, $exceptArray, 0);
                if (
                    // for unknown reasons, if we have not ran fread() on the socket at least once,
                    // stream_select() is UNRELIABLE on Ubuntu24.04 and return 0 even if there is data to read.
                    !$this->hasBeenFreadInitialized
                    // > 0 mean that there is incoming data
                    || $numChangedStreams > 0
                    // false means we were unable to check for incoming data,
                    // false is no guarantee that there actually isn't any data to read
                    // (but false DOES mean that something is fucked up...)
                    || $numChangedStreams === false
                ) {
                    $result = \fread($this->socket, $length);
                    $this->hasBeenFreadInitialized = true;
                } else {
                    $result = false;
                }
                if ($makeBlockingAfterRead) {
                    \stream_set_blocking($this->socket, true);
                    $makeBlockingAfterRead = false;
                }

                if (false === $result) {
                    return $buffer;
                }

                $buffer .= $result;

                // feof means socket has been closed
                if (\feof($this->socket)) {
                    $this->disconnect();

                    return $buffer;
                }

                $continue = false;

                if (1 === \strlen($result)) {
                    // Workaround Chrome behavior (still needed?)
                    $continue = true;
                }

                if (\strlen($result) === $length) {
                    $continue = true;
                }

                // Continue if more data to be read
                $metadata = \stream_get_meta_data($this->socket);

                /** @phpstan-ignore-next-line */
                if (isset($metadata['unread_bytes'])) {
                    if (!$metadata['unread_bytes']) {
                        // stop it, if we read a full message in previous time
                        $continue = false;
                    } else {
                        $continue = true;
                        // it makes sense only if unread_bytes less than DEFAULT_RECEIVE_LENGTH
                        if ($length > $metadata['unread_bytes']) {
                            // http://php.net/manual/en/function.stream-get-meta-data.php
                            // 'unread_bytes' don't describes real length correctly.
                            // $length = $metadata['unread_bytes'];

                            // Socket is a blocking by default. When we do a blocking read from an empty
                            // queue it will block and the server will hang. https://bugs.php.net/bug.php?id=1739
                            \stream_set_blocking($this->socket, false);
                            $makeBlockingAfterRead = true;
                        }
                    }
                }
            } while ($continue);

            return $buffer;
        } finally {
            if ($this->socket && !\feof($this->socket) && $makeBlockingAfterRead) {
                \stream_set_blocking($this->socket, true);
            }
        }
    }

[bsienn-khan]

Amazing, I wonder why this happened in our case and not effecting others. Now how would I use this solution? Wait for pr or just use your function for time being?