feat(runtime): add a JavaScript evaluation API with named arguments, keeping the old one with warnings

CHANGELOG.md

@@ -104,6 +104,9 @@
   loaded (on CDP's `Network.loadingFinished`, not `Network.responseReceived`, so `exchange.response.body` is always
   available), yielding the request's `Network::Exchange`. Doesn't require `network.intercept` to be set up, and is
   never fired for requests that fail to load [#294]
+- `#evaluate`/`#execute` take named arguments that become the script's parameters, `page.evaluate("a + b", a: 1,
+  b: 2)`, a function declaration as the script, `await`, and `timeout:`
+- `#evaluate_handle` returning a `Ferrum::RemoteObject`, and `Node#execute`/`Node#evaluate_handle`
 - `Ferrum::Browser#quit`/`Ferrum::Browser::Process#stop` accept `wait: false` to return immediately and run process
   killing and user-data-directory cleanup on a background thread instead of blocking; the call returns the `Thread`
   so callers can `#join` it if they need cleanup to have finished, e.g. before process exit or before reusing a
@@ -119,6 +122,9 @@
 - `Ferrum::Page::Stream#stream` now closes the CDP stream handle (`IO.close`) once it's been fully read, so streams
   opened for `Ferrum::Browser#pdf` and `Ferrum::Page::Tracing#record` no longer keep their backing storage alive in
   the browser; a failed `IO.close` is raised to the caller.
+- `#evaluate_async`, `#evaluate_func` and `#evaluate_on` are deprecated, and scripts whose result will change in the upcoming
+  0.21 warn with a rewrite, see [upgrading](docs/13-javascript.md#upgrading-to-021)
+- `Ferrum::ScriptTimeoutError` is a `Ferrum::TimeoutError`
 
 ### Fixed
 - An out-of-process iframe, e.g. Chrome's PDF viewer for an embedded PDF, appearing while a web worker was connected

README.md

@@ -85,6 +85,59 @@ JS
 browser.quit
 ```
 
+Pass arguments by name, so the script can use parameters instead of digging
+through `arguments[0]`:
+
+```ruby
+page.evaluate("a + b", a: 1, b: 2) # => 3
+page.evaluate("el.getAttribute(name)", el: page.at_css("a"), name: "href")
+```
+
+A script that starts with a function or arrow declaration is used as-is, which
+is how you run more than one statement:
+
+```ruby
+page.evaluate(<<~JS, a: 1, b: 2)
+  function(a, b) {
+    const sum = a + b;
+    return sum * 3;
+  }
+JS
+```
+
+`await` works directly, and a function declaration's returned promise is
+awaited:
+
+```ruby
+page.evaluate("await fetch(url).then(r => r.text())", url: "/api")
+```
+
+Pass `timeout:` (seconds, defaulting to the page timeout) to bound how long the
+script may take before `Ferrum::ScriptTimeoutError` is raised, or `0` to let it
+run for as long as it takes. `timeout:` and `args:` are the only reserved
+keywords; use `args:` when a JavaScript parameter needs one of those names:
+
+```ruby
+page.evaluate("timeout * 2", args: { timeout: 21 }) # => 42
+```
+
+Use `#execute` when you only want the side effects, `#evaluate_handle` when you
+want to keep a value in the browser and pass it back in later, and
+`Ferrum::Node#evaluate` to run a script with `this` bound to an element:
+
+```ruby
+page.execute("window.scrollBy(0, 100)") # => true
+
+list = page.evaluate_handle("document.querySelectorAll('li')")
+page.evaluate("Array.from(nodes).map(n => n.textContent)", nodes: list)
+
+page.at_css("input").evaluate("this.value")
+```
+
+Scripts written for the old API keep working, and Ferrum warns about any whose
+result will change in the next release, 0.21, with a suggested rewrite. See
+[upgrading to 0.21](docs/13-javascript.md#upgrading-to-021).
+
 Do any mouse movements you like:
 
 ```ruby

docs/13-javascript.md

@@ -4,43 +4,78 @@ sidebar_position: 13
 
 # JavaScript
 
-#### evaluate(expression, \*args)
+#### evaluate(expression, \*args, timeout: nil, \*\*named)
 
-Evaluate and return result for given JS expression
+Evaluate JavaScript and return the result, converted to Ruby.
 
-* expression `String` should be valid JavaScript
-* args `Object` you can pass arguments, though it should be a valid `Node` or a
-  simple value.
+* expression `String` a JavaScript expression, or a function/arrow declaration, which is used as-is and is how you
+  run more than one statement
+* args positional arguments, reachable as `arguments[0]` and up. Can't be mixed with named ones
+* timeout `Numeric` seconds to wait for the script and any promise it returns, defaults to the page timeout, `0`
+  waits for as long as it takes. Raises `Ferrum::ScriptTimeoutError`, a `Ferrum::TimeoutError`, when exceeded
+* named named arguments, which become the script's parameters. A `Ferrum::Node` arrives as the live element. Use
+  `args: { timeout: 1 }` for a parameter named `timeout` or `args`
 
 ```ruby
-page.evaluate("[window.scrollX, window.scrollY]")
+page.evaluate("[window.scrollX, window.scrollY]") # => [0, 0]
+page.evaluate("a + b", a: 1, b: 2) # => 3
+page.evaluate("el.getAttribute(name)", el: page.at_css("a"), name: "href")
+page.evaluate("await fetch(url).then(r => r.status)", url: "/") # => 200
+page.evaluate(<<~JS, c: 3) # => 9
+  function(c) {
+    const sum = 1 + 2;
+    return sum * c;
+  }
+JS
 ```
 
-#### evaluate_async(expression, wait_time, \*args)
-
-Evaluate asynchronous expression and return result
+#### evaluate_handle(expression, \*args, timeout: nil, \*\*named) : `Ferrum::RemoteObject`
 
-* expression `String` should be valid JavaScript
-* wait_time How long we should wait for Promise to resolve or reject
-* args `Object` you can pass arguments, though it should be a valid `Node` or a
-  simple value.
+Same as `evaluate`, but keeps the value in the browser and returns a reference you can pass back in as an argument.
+Call `#value` to convert it to Ruby, and `#release` to free it.
 
 ```ruby
-page.evaluate_async(%(arguments[0]({foo: "bar"})), 5) # => { "foo" => "bar" }
+list = page.evaluate_handle("document.querySelectorAll('li')")
+page.evaluate("Array.from(nodes).map(n => n.textContent)", nodes: list)
 ```
 
-#### execute(expression, \*args)
+#### execute(expression, \*args, timeout: nil, \*\*named) : `Boolean`
+
+Run JavaScript for its side effects. The script is used as a function body, so statements need no wrapping.
 
-Execute expression. Doesn't return the result
+```ruby
+page.execute("window.scrollBy(0, 100)") # => true
+```
 
-* expression `String` should be valid JavaScript
-* args `Object` you can pass arguments, though it should be a valid `Node` or a
-  simple value.
+`Ferrum::Node` has the same three methods, with `this` bound to the element. Arrow functions don't get `this`, so use a
+`function` declaration when you need it.
 
 ```ruby
-page.execute(%(1 + 1)) # => true
+page.at_css("input").evaluate("this.value")
+page.at_css("input").execute("this.value = text", text: "hello")
 ```
 
+#### Upgrading to 0.21
+
+The next release, Ferrum 0.21, finishes the move to the API above. Until then, scripts written for the old API run
+exactly as before, and Ferrum warns, pointing at your code, about any whose result will change. `evaluate_async`,
+`evaluate_func` and `evaluate_on` are deprecated and will be removed in 0.21. Set
+`FERRUM_DEPRECATION_WARNINGS=raise` to turn the warnings into errors while migrating, or `0` to silence them.
+
+| Warning | In 0.20 | From 0.21 | Rewrite |
+|---|---|---|---|
+| Keyword arguments read through `arguments` | one object in `arguments[0]` | named parameters | `evaluate("options.foo", options: { foo: 1 })` |
+| Mixing positional and keyword arguments | keywords appended as an object | `ArgumentError` | pass the object in braces, or name every argument |
+| More than one expression, `"a(); b"` | returns `a()`, skips the rest | `SyntaxError` | `evaluate("function() { a(); return b }")` |
+| Never runs, a line break before the script | `nil`, nothing runs | runs and returns its value | check it's safe to run, or remove it |
+| Returns a promise | `{}`, not awaited | awaited, raises if rejected | prefix with `void` to keep it from waiting |
+| Returns an object with a `then` method | the object | awaited like a promise | wrap it, `({ value: object })` |
+| Throws something that isn't an `Error` | the thrown value | `Ferrum::JavaScriptError` | return the value instead |
+| `Node#evaluate` returns a node, nodes or a `Date` | `{}` | `Ferrum::Node`, or the date's string | check the code using the result |
+| `evaluate_async(expression, wait, *args)` | deprecated | removed | `evaluate("await …", timeout: wait)` |
+| `evaluate_func(function, *args, on:)` | deprecated | removed | `evaluate(function, a: 1)`, or `node.evaluate(function)` |
+| `evaluate_on(node:, expression:)` | deprecated | removed | `node.evaluate(expression)` |
+
 #### evaluate_on_new_document(source, \*\*options) : `String`
 
 Runs JavaScript in every new document of the page and its in-process frames, before the document's own scripts.
@@ -62,7 +97,7 @@ JS
 #### isolated_world(name) : `IsolatedWorld`
 
 Returns the frame's isolated world with this name, its own JavaScript globals over the frame's DOM, so the page's
-scripts can't see or override what runs there. It has `evaluate`, `evaluate_async`, `execute` and `evaluate_func`,
+scripts can't see or override what runs there. It has `evaluate`, `evaluate_handle` and `execute`,
 takes and returns nodes, and comes back empty in every new document. On a page it's the main frame's world, use
 `frame.isolated_world(name)` for an iframe. It's the same world `evaluate_on_new_document` runs in with
 `world_name:`.

docs/21-workers.md

@@ -36,7 +36,7 @@ Returns service worker targets registered in this context. Unlike `workers`, the
 are not connected to -- call `attach_target` first if you want to interact with one.
 
 ```ruby
-page.evaluate_async(%(navigator.serviceWorker.register("/sw.js").then(arguments[0])), 5)
+page.evaluate(%(await navigator.serviceWorker.register("/sw.js")))
 
 sleep 0.1
 
@@ -91,7 +91,7 @@ Same API as `page.console` -- see [JavaScript](/docs/ferrum/javascript#console).
 worker.console.map(&:text) # => ["from worker"]
 ```
 
-#### evaluate / execute / evaluate_async
+#### evaluate / execute / evaluate_handle
 
 Same API as on `Page`/`Frame` -- see [JavaScript](/docs/ferrum/javascript). A worker has a single execution context,
 so there's no frame to target.

lib/ferrum.rb

@@ -7,8 +7,10 @@
 require "ferrum/utils/platform"
 require "ferrum/utils/elapsed_time"
 require "ferrum/utils/attempt"
+require "ferrum/utils/deprecate"
 require "ferrum/errors"
 require "ferrum/browser"
+require "ferrum/remote_object"
 require "ferrum/node"
 
 #

lib/ferrum/browser.rb

@@ -36,7 +36,8 @@ class Browser
                 screenshot pdf mhtml viewport_size device_pixel_ratio
                 start_screencast stop_screencast
                 frames frame_by main_frame
-                evaluate evaluate_on evaluate_async execute evaluate_func isolated_world
+                evaluate evaluate_handle execute
+                evaluate_on evaluate_async evaluate_func isolated_world
                 add_script_tag add_style_tag bypass_csp
                 evaluate_on_new_document remove_script_on_new_document
                 on position position=

lib/ferrum/client.rb

@@ -209,26 +209,29 @@ def command(method, async: false, timeout: nil, **params)
     #
     # @param [Numeric, nil] timeout
     #   How long to wait for the browser to take the message, and then for its
-    #   response. Defaults to `protocol_timeout`.
+    #   response. Defaults to `protocol_timeout`. `Float::INFINITY` waits for
+    #   the response for as long as it takes, still giving the browser
+    #   `protocol_timeout` to take the message.
     #
     # @return [Boolean, Hash]
     #   `true` when sent asynchronously, otherwise the parsed `"result"`
     #   from the response.
     #
     def send_message(message, async:, timeout: nil)
       timeout ||= protocol_timeout
+      write_timeout = timeout.infinite? ? protocol_timeout : timeout
 
       if async
-        @ws.send_message(message, timeout)
+        @ws.send_message(message, write_timeout)
         true
       else
         raise DeadBrowserError if @ws.messages.closed?
 
         pending = Concurrent::IVar.new
         @pendings[message[:id]] = pending
         begin
-          @ws.send_message(message, timeout)
-          data = pending.value!(timeout)
+          @ws.send_message(message, write_timeout)
+          data = pending.value!(timeout.infinite? ? nil : timeout)
         ensure
           @pendings.delete(message[:id])
         end

lib/ferrum/errors.rb

@@ -87,7 +87,7 @@ def awaited
   end
 
   # Raised when an evaluated script takes too long to return a value.
-  class ScriptTimeoutError < Error
+  class ScriptTimeoutError < TimeoutError
     #
     # Explains that the evaluated script timed out.
     #
@@ -216,7 +216,7 @@ class JavaScriptError < BrowserError
 
     def initialize(response)
       @class_name, @message = response["exception"]&.values_at("className", "description")
-      @message ||= response["text"]
+      @message ||= [response["text"], response.dig("exception", "value")&.inspect].compact.join(" ")
       @stack_trace = response["stackTrace"]
       super(response.merge("message" => @message))
     end

lib/ferrum/frame.rb

@@ -167,12 +167,11 @@ def parent
     #   frame.body # => <html><head></head><body><p>lol</p></body></html>
     #
     def content=(html)
-      evaluate_async(%(
+      execute(<<~JS, html: html)
         document.open();
-        document.write(arguments[0]);
+        document.write(html);
         document.close();
-        arguments[1](true);
-      ), @page.timeout, html)
+      JS
       @page.document_node_id
     end
     alias set_content content=