JSON::Class
is now released. The previous post explains why does this worth a note.]]>WWW::GCloud
modules for accessing Google Cloud services. Their REST API is, apparently, JSON-based. So, I made use of the existing JSON::Class
. Unfortunately, it was missing some features critically needed for my work project. I implemented a couple of workarounds, but still felt like it’s not the way it has to be. Something akin to LibXML::Class
would be great to have…
There was a big “but” in this. We already have XML::Class
, LibXML::Class
, and the current JSON::Class
. All are responsible for doing basically the same thing: de-/serializing classes. If I wanted another JSON serializer then I had to take into account that JSON::Class
is already taken. There are three ways to deal with it:
JSON::Class
and re-implement it as a backward-incompatible version.The first two options didn’t appeal to me. The third one is now about to happen.
I expect it to be a stress-test for Raku ecosystem as, up to my knowledge, it’s going to be the first case where two different modules share the same name but not publishers.
As a little reminder:
JSON::Class:auth<zef:jonathanstowe>
in their dependencies and, perhaps, in their use
statement.JSON::Class:auth<zef:vrurg>
.There is still some time before I publish it because the documentation is not ready yet.
Let’s 🤞🏻.
]]>No, not this way. A technology must be easy to start with, but also be easy in accessing its advanced or fine-tunable features. Let’s have an example of the former.
This post is a quick hack, no proof-reading or error checking is done. Please, feel free to report any issue.
Part of my ongoing project is to deal with JSON data and deserialize it into Raku classes. This is certainly a task
for JSON::Class
. So far, so good.
The keys of JSON structures tend to use lower camel case which is OK, but we like
kebabing in Raku. Why not, there is
JSON::Name
. But using it:
There are roles. At the point I came to the final solution I was already doing something like1:
class SomeStructure does JSONRecord {...}
Then there is AttrX::Mooish
, which is my lifevest on many occasions:
use AttrX::Mooish;
class Foo {
has $.foo is mooish(:alias<bar>);
}
my $obj = Foo.new: bar => "the answer";
say $obj.foo; # the answer
Apparently, this way it would still be a lot of manual interaction with aliasing, and that’s what I was already doing for a while until realized that there is a bettter way. But be back to this later…
And, eventually, there are traits and MOP.
That’s the easiest part. What I want is to makeThisName
look like make-this-name
. Ha, big deal!
unit module JSONRecord::Utils;
our sub kebabify-attr(Attribute:D $attr) {
if $attr.name ~~ /<.lower><.upper>/ {
my $alias = (S:g/<lower><upper>/$<lower>-$<upper>/).lc given $attr.name.substr(2);
...
}
}
I don’t export the sub because it’s for internal use mostly. Would somebody need it for other purposes it’s a rare case where a long name like JSONRecord::Utils::kebabify-attr($attr)
must not be an issue.
The sub is not optimal, it’s what I came up with while expermineting with the approach. The number of method calls and regexes can be reduced.
I’ll get back later to the yada-yada-yada up there.
Now we need a bit of MOP magic. To handle all attributes of a class we need to iterate over them and apply the aliasing. The first what comes to mind is to use role body because it is invoked at the early class composition times:
unit role JSONRecord;
for ::?CLASS.^attributes(:local) -> $attr {
# take care of it...
}
Note the word “early” I used above. It actually means that when role’s body is executed there are likely more roles waiting for their turn to be composed into the class. So, there are likely more attributes to be added to the class.
But we can override Metamodel::ClassHOW
compose_attributes
method of our target ::?CLASS
and rest assured no one would be missed:
unit role JSONRecordHOW;
use JSONRecord::Utils;
method compose_attributes(Mu \obj, |) {
for self.attributes(obj, :local) -> $attr {
# Skip if it already has `is mooish` trait applied – we don't want to mess up with user's intentions.
next if $attr ~~ AttrX::Mooish::Attribute;
JSONRecord::Utils::kebabify-attr($attr);
}
nextsame
}
Basically, that’s all we currently need to finalize the solution. We can still use role’s body to implement the key elements of it:
unit role JSONRecord;
use JSONRecordHOW;
unless ::?CLASS.HOW ~~ JSONRecordHOW {
::?CLASS.HOW does JSONRecordHOW;
}
Job done! Don’t worry, I haven’t forgot about the yada-yada-yada above!
But…
The original record role name itself is even longer than JSONRecord
, and it consists of three parts. I’m lazy. There are a lot of JSON structures and I want less typing per each. A trait? is jrecord
?
unit role JSONRecord;
multi sub trait_mod:<is>(Mu:U \type, Bool:D :$jrecord) is export {
unless type.HOW ~~ JSONRecordHOW {
type.HOW does JSONRecordHOW
type.^add_role(::?ROLE);
}
}
Now, instead of class SomeRecord does JSONRecord
I can use class SomeRecord is jrecord
. In the original case the win is even bigger.
There is absolutely nothing funny about it. Just a common way to keep a reader interested!
Seriously.
The reason for the yada in that snippet is to avoid a distraction from the primary purpose of the example. Here is what is going on there:
I want AttrX::Mooish
to do the dirty work for me. Eventually, what is needed is to apply the is mooish
trait as shown above. But the traits are just subs. Therefore all is needed now is to:
&trait_mod:<is>($attr, :mooish(:$alias));
Because this is what Raku does internally when encounters is mooish(:alias(...))
. The final version of the kebabifying sub is:
our sub kebabify-attr(Attribute:D $attr) {
if $attr.name ~~ /<.lower><.upper>/ {
my $alias = (S:g/<lower><upper>/$<lower>-$<upper>/).lc given $attr.name.substr(2);
&trait_mod:<is>($attr, :mooish(:$alias));
}
}
Since the sub is used by the HOW above, we can say that the &trait_mod<is>
would be called at compile time2.
Now, it used to be:
class SomeRecord does JSONRecord {
has $.aLongAttrName is mooish(:alias<a-long-attr-name>);
has $.shortname;
}
Where, as you can see, I had to transfer JSON key names to attribute names, decide where aliasing is needed, add it, and make sure no mistakes were made or attributes are missed.
With the above rather simple tweaks:
class SomeRecord is jrecord {
has $.aLongAttrName;
has $.shortname;
}
Job done.
Before I came down to this solution I’ve got 34 record classes implemented using the old approach. Some are little, some are quite big. But it most certainly could’ve taken much less time would I have the trait at my disposal back then…
]]>It’s likely to take a long before I can write another.
]]>Answers to the question would depend on particular circumstances of the code where this functionality is needed. One
would be about using MOP methods like .^lookup
, the other is to use method name and indirect resolution on invocant:
self."$method-name"(...)
. Both are the most useful, in my view. But sometimes declaring a method as our
can be
helpful too:
class Foo {
our method bar {}
}
say Foo::<&bar>.raku;
Just don’t forget that this way we always get the method of class Foo
, even if a subclass overrides method bar
.
And I gave up. With a few cut-outs and some titles added the video is now available. Enjoy and sorry for the glitches!
]]>OK, not exactly. I hate the way they abuse it, especially in the corporate world. XML, and Java (or Python lately), and no alternatives allowed. Often, where even CSV would do better – they stick in an XML-based bloatware and get always happy about it!
But then it turns out that I have this bunch of very different PDFs on my hands, from different sources, produced by various software tools, but with one thing in common: they all contain data I must pull in and sort out. Lucky me, nobody puts limits on what this has to be implemented in: “Raku? What is it? Whatever, just make it work!” Fantastic!
Next turn, what do we have for reading PDFs? Oh, oops… Can I convert them into something different? Sure, you can! Would you be happy about XML? Er… Well, yes, I suppose.
In a while, along with the PDFs, an XLSX spreadsheet pops up. And you all know what is it internally… Moreover, back
then it became apparent to me that Spreadsheet::XLSX
lacks support
for some key features of the format and I came up with a couple of PRs to implement them. Along the lines I basically
developed a small core for de-/serializing XML into Raku objects. It’s limited and only sufficient to serve the needs of
XLSX parsing, but it felt like having some interesting potential. Especially with regard to the tasks I already had on
my hands.
Yet, before doing something as stupid as starting a new project, I checked around first and, apparently, stumbled upon
XML::Class
. Unfortunately, the
XML
module, it is based upon, proved to be too slow for the files
I’m dealing with, LibXML
is doing way better. Still, XML::Class
provided me
with a couple of great ideas.
And here we are today: welcome the LibXML::Class
module! A swiss army
knife of XML serialization for The Raku language.
First of all, the principle I’m trying to follow any time something new is planned: make it easy to start with, yet make it very capable:
use LibXML::Class;
class Record is xml-element {
has Str $.field1;
has Int:D $.field2 is required;
}
say Record.new(
field1 => 'The Answer',
field2 => 42
).to-xml;
say Record.new( field2 => 12 ).to-xml;
How much easier could this ever be? There is even better example in the repository, but I’m not including it here now because I have different plans for it.
Ok, this is about simplicity. What about capability? The full list of module’s features can be found in its README and the manual explains them in details, though due to lack of time no proofreading has been done for it and errors of different kinds are guaranteed. That’s why I tried to cover most important topics with examples. In this post I’d cover just the most important ones.
Say, we have a huge XML with complex structure. Full and immediate deserialization of it would result in hundreds or thousands of instances of Raku classes created. No fast parser would be of much help here because of the time it takes to run all the constructors of each and every object. If one needs just one attribute of a single records somewhere in the structure it’d be definitely stupid waste of computing time and memory and, worst of all, end user patience.
This is not our way. LibXML::Class
doesn’t deserialize until it is really necessary. Consider an
example from the repository:
use v6.e.PREVIEW;
use LibXML::Class;
class Record is xml-element {
has Str:D $.data is required;
submethod TWEAK {
say "+ record";
}
}
class Root is xml-element {
has Record:D $.record is required;
submethod TWEAK {
say "+ root";
}
}
my $root = Root.new: record => Record.new(:data("some data"));
say "--- deserializing";
my $root-copy = Root.from-xml: $root.to-xml.Str;
say "--- deserialized";
say $root-copy.record;
say "--- all done";
Running it would result in an output like this:
+ record
+ root
--- deserializing
+ root
--- deserialized
+ record
Record.new(data => "some data", xml-name => "Record")
--- all done
The first two lines are rather understandable: we create a record, then the root object. Hence the prints from their
constructors. But starting from ’— deserializing’ line the events get more interesting twist. We only see an output
from Root
’s constructor, but there is nothing from Record
. That is because at this point $.record
is not
initialized yet. LibXML::Class
is using AttrX::Mooish
to turn the
attribute in a lazy one as if somebody applied is mooish(:lazy<xml-deserialize-attr>)
trait to it. The effect of this
action is visible right after the end of deserialization is reported with ’— deserialized’ line. There you can see
’+ record’ from Record
’s TWEAK
submethod first, and only after that there is a gist of the record object itself.
Both are easily pinpointed to the say $root-copy.record
line, where a read from $.record
resulted in the object
being eventually deserialized and made ready for use.
Now, imagine that the Record
itself has sub-records, and sub-sub-records, and there are lots and lots of them. But
your code doesn’t waste time on instantiating – unless explicitly requested to do so as, apparently, this behavior can
be disabled if necessary. Moreover, it can be triggered on or off at individual level per attribute.
This functionality is not activated implicitly for basic-type attributes like strings, numerics, etc. But one can enforce it per-attribute, if this is considered helpful
Working on Spreadsheet::XLSX
introduced me to such pretty curious entity as XML sequence. From Raku’s perspective it
would be a Positional
, and an Iterable
; but neither a List
nor a Seq
. Well, in theory it is possible to map it
into one, but that’d be rather tricky and unnatural. Here is the most challenging features of a sequence:
Perhaps I miss something here, but even these three points make it somewhat special.
Sure, with certain amount of patience and obstinacy, one could implement them as arrays, but here come one barely solvable problem: an array attribute would still be deserialized as a whole simply because there are no lazy arrays in Raku!
Here comes a solution (see another example):
use v6.e.PREVIEW;
use LibXML::Class;
class Ref is xml-element<ref> {
has Str:D $.ISBN is required is xml-element;
has Int:D $.page is required is xml-element;
submethod TWEAK {
say "+ ref for ", $!ISBN;
}
}
class Index is xml-element( 'index',
:sequence( Ref, :idx(Int:D) ) )
{
has Str:D $.title is required;
}
my $index = Index.new: title => "Experimental";
$index.push: 42;
$index.push: Ref.new(:ISBN<1-2-FAKE>, :page(10));
$index.push: Ref.new(:ISBN<3-4-MOCKED>, :page(1001));
say "--- deserializing";
my $index-copy = Index.from-xml: $index.to-xml.Str;
say "--- deserialized";
say $index-copy[1];
say "--- all done";
Along the lines, the sample also demonstrates how advanced capabilities of LibXML::Class
get activated when
necessary.
Anyway, running this would result in the following output:
+ ref for 1-2-FAKE
+ ref for 3-4-MOCKED
--- deserializing
--- deserialized
+ ref for 1-2-FAKE
Ref.new(ISBN => "1-2-FAKE", page => 10, xml-name => "ref")
--- all done
And, again, we observe laziness in action! As only the single item on the sequence is read from – only single output is
produced by the TWEAK
. There is a difference to the attributes though: XML sequence is totally and unexceptionally
lazy. No sequence item is deserialized until read, not even the basic type ones.
Now, let’s get back to where it started. In an XLSX worksheet rows are sequence elements (items in terms of
LibXML::Class
Raku representation); same apply to individual cells. Now, imagine full deserialization of a sheet
consisting of thousand lines with hundreds of columns! Nah, gimme a break… Of course, that would still mean full
parsing of the XML, but, unfortunately, it’s unavoidable cost. Yet, it doesn’t mean that there gonna be piles of
LibXML::Node
objects scattered all around your RAM! Fortunately for us, most of the work would be done at the low
level by libxml
and only pulled up into the Raku land when necessary. In other words, these ops are also mostly lazy.
This is where I both love it and hate it. Use of namespaces in XML helps resolve so many problems that often times XML is the only answer to complex problems. Though in my view a way less verbose format could’ve been developed for these tasks, but it’s too late for any discussions now. Hence my only complain here is about using XML where there are more appropriate formats that would do better.
Anyway, my goal was to cover as many different combinations of using namespaces as I could. Considering that we have
default namespaces (these defined with xmlns="..."
), and we have prefixes, and we have priorities, inheritance,
override, and perhaps some other things I forget about; and there are rules on how they apply and match; and that the
way one see and use them with Raku objects must look and feel the same as for XML (or, at least, for LibXML
implementation of the standard); so, considering all the above, in the retrospective, I’m not surprised that more than
80% of code development time has been spent on namespacing. Parts of the code underwent like 5-10 rewrites – basically,
the count has been lost long ago…
But it’s definitely worth it. Just by looking at the module’s SYNOPSIS you can see how simple is it to declare and refer to namespaces!
Then you start reading the manual.
Then you come down to see an example of deriving namespaces.
Then there is an example of imposing namespaces.
And only then it gets apparent how convoluted namespacing could be. Yet, we can handle if not all possible variations of it, but a significant subset, most certainly!
BTW, another feature I wouldn’t focus upon but wanna mention anyway is
XML:any technique, which
is heavily based upon namespaces. This is an idea I borrowed from XML::Class
but gave it some extra capabilities,
especially in the area of XML sequence items.
Examples are intentionally omitted in this section due to their rather bloated sizes.
Here comes real magic!
When I encountered LibXML
, aside of its speed, what made me attracted (let’s avoid emotionally attached term,
though…) is its XPath-based findnodes
. And when I came down to the idea of LibXML::Class
the method was one of the
two most significant reasons I wanted the module to exists in first place.
Well, you know what? I nearly forgot to implement it, after all. The namespaces, you know: they sucked every last bit of energy out of me. But, down with them…
Can you spot a catch here? I’ll give a hint: laziness. It wouldn’t be a big deal to map a LibXML::Node
to its
deserialization because there is the unique-key
method which lets us keep track of objects. But what if there is no
object to track yet? What if the node we found is so deeply nested in the source XML that not only there is no
deserialization for it, but for a couple of its parent nodes too?
Solved. The feature can be observed in action in modified version of SYNOPSIS code. Tests 200-pml-parser.rakutest and 150-find.rakutest are even better in demonstrating the feature, but they’re apparently harder to read. 200-pml-parser.rakutest is specifically focused on searching for undeserialized yet nodes.
I’m once again avoiding any full samples here. They’d be too big. Just to give you an impression on how it works, here is the single line which would be at the core of most searches:
my LibXML::Class::XML:D @deserializations = $root.xml-findnodes(q«//*[@idx = 3002]»);
This is all needed to find deserializations for all XML elements with idx
attribute set to 3002.
Wait, don’t go! Just one another line and we’re almost done here:
my Str:D @names = $root.xml-findnodes(q«//*[@idx = 3002]/@name»);
This is how find attributes name
of our elements. So, let’s say there is something like this in our Raku:
role Named is xml-element {
has Str:D $.name is xml-attribute;
}
role Indexed is xml-element {
has UInt:D $.idx is xml-attribute;
}
class Record is xml-element<rec> does Named does Indexed {
has SubRecord $.sr is xml-element<subrec>;
}
The by adding @name to the XPath we’d get $.name
of the role Named
. If there are other classes consuming it and
deserialized from the same XML then we gonna get their attributes too, perhaps. Surely, it depends on $.idx
values.
Pardon? Haven’t I told about roles? Oh, my bad! Well, you see them supported. As well as subclassing. Let’s not focus on this.
What’s more interesting is that search works with object cloning. It means that even there is 100% probability that there is single XML element to be found, a sequence would always be returned for a particular XPath expression. Because if a deserialization gets cloned the first thing the newborn copy does is registers itself with the object registry.
And, sure enough, if search is not needed then turning it off altogether would spare you some memory and processing times.
Now I say myself: stop! Or a post would turn into a secondary manual. Before we say each other “see ya!” I would have one single request to you: if this module ever makes you want to use it – please, make sure it’s not for a manually editable config of your application! XML is great when used properly; and “properly” means to me: read and written by and only by code, never by human eyes and human hands!
]]>class Foo::Bar {
}
And there is another class Baz
for which we want it to be coercible into
Foo::Bar
. No problem!
class Baz {
method Foo::Bar() { Foo::Bar.new }
}
Now we can do:
sub foo(Foo::Bar() $v) { say $v }
foo(Baz.new);
My recent tasks are spinning around concurrency in one way or another. And where the concurrency is there are locks. Basically, introducing a lock is the most popular and the most straightforward solution for most race conditions one could encounter in their code. Like, whenever an investigation results in a resolution that data is being updated in one thread while used in another then just wrap both blocks into a lock and be done with it! Right? Are you sure?
They used to say about Perl that “if a problem is solved with regex then you got
two problems”. By changing ‘regex’ to ‘lock’ we shift into another domain. I
wouldn’t discuss interlocks here because it’s rather a subject for a big CS
article. But I would mention an issue that is possible to stumble upon in a
heavily multi-threaded Raku application. Did you know that Lock
, Raku’s most
used type for locking, actually blocks its thread? Did you also know that
threads are a limited resource? That the default ThreadPoolScheduler
has a
maximum, which depends on the number of CPU cores available to your system? It
even used to be a hard-coded value of 64 threads a while ago.
Put together, these two conditions could result in stuck code, like in this example:
BEGIN PROCESS::<$SCHEDULER> = ThreadPoolScheduler.new: max_threads => 32;
my Lock $l .= new;
my Promise $p .= new;
my @p;
@p.push: start $l.protect: { await $p; };
for ^100 -> $idx {
@p.push: start { $l.protect: { say $idx } }
}
@p.push: start { $p.keep; }
await @p;
Looks innocent, isn’t it? But it would never end because all available threads
would be consumed and blocked by locks. Then the last one, which is supposed to
initiate the unlock, would just never start in first place. This is not a bug in
the language but a side effect of its architecture. I had to create
Async::Workers
module a while ago to solve a task which was hit by this issue.
In other cases I can replace Lock
with Lock::Async
and it would just work.
Why? The answer is in the following section. Why not always Lock::Async
?
Because it is rather slow. How much slower? Read on!
Lock
vs. Lock::Async
What makes these different? To put it simple, Lock
is based on system-level
routines. This is why it is blocking: because this is the default system
behavior.
Lock::Async
is built around Promise
and await
. The point is that in Raku
await
tries to release a thread and return it back into the scheduler pool,
making it immediately available to other jobs. So does Lock::Async
too: instead
of blocking, its protect
method enters into await
.
BTW, it might be surprising to many, but lock
method of Lock::Async
doesn’t
actually lock by itself.
There is one more way to protect a block of code from re-entering. If you’re well familiar with atomic operations then you’re likely to know about it. For the rest I would briefly explain it in this section.
Let me skip the part about the atomic operations as such, Wikipedia has it. In particular we need CAS (Wikipedia again and Raku implementation). In a natural language terms the atomic approach can be “programmed” like this:
Note that 1 and 3 are both atomic ops. In Raku code this is expressed in the following slightly simplified snippet:
my atomicint $lock = 0; # 0 is unlocked, 1 is locked
while cas($lock, 0, 1) == 1 {} # lock
... # Do your work
$lock ⚛= 0; # unlock
Pretty simple, isn’t it? Let’s see what are the specs of this approach:
Lock
Item 2 is speculative at this moment, but guessable. Contrary to Lock
, we
don’t use a system call but rather base the lock on a purely computational
trick.
Item 3 is apparent because even though Lock
doesn’t release it’s thread for
Raku scheduler, it does release a CPU core to the system.
As I found myself in between of two big tasks today, I decided to make a pause
and scratch the itch of comparing different approaches to locking. Apparently,
we have three different kinds of locks at our hands, each based upon a specific
approach. But aside of that, we also have two different modes of using them. One
is explicit locking/unlocking withing the protected block. The other one is to
use a wrapper method protect
, available on Lock
and Lock::Async
. There is
no data type for atomic locking, but this is something we can do ourselves and
have the method implemented the same way, as Lock
does.
Here is the code I used:
constant MAX_WORKERS = 50; # how many workers per approach to start
constant TEST_SECS = 5; # how long each worker must run
class Lock::Atomic {
has atomicint $!lock = 0;
method protect(&code) {
while cas($!lock, 0, 1) == 1 { }
LEAVE $!lock ⚛= 0;
&code()
}
}
my @tbl = <Wrkrs Atomic Lock Async Atomic.P Lock.P Async.P>;
my $max_w = max @tbl.map(*.chars);
printf (('%' ~ $max_w ~ 's') xx +@tbl).join(" ") ~ "\n", |@tbl;
my $dfmt = (('%' ~ $max_w ~ 'd') xx +@tbl).join(" ") ~ "\n";
for 2..MAX_WORKERS -> $wnum {
$*ERR.print: "$wnum\r";
my Promise:D $starter .= new;
my Promise:D @ready;
my Promise:D @workers;
my atomicint $stop = 0;
sub worker(&code) {
my Promise:D $ready .= new;
@ready.push: $ready;
@workers.push: start {
$ready.keep;
await $starter;
&code();
}
}
my atomicint $ia-lock = 0;
my $ia-counter = 0;
my $il-lock = Lock.new;
my $il-counter = 0;
my $ila-lock = Lock::Async.new;
my $ila-counter = 0;
my $iap-lock = Lock::Atomic.new;
my $iap-counter = 0;
my $ilp-lock = Lock.new;
my $ilp-counter = 0;
my $ilap-lock = Lock::Async.new;
my $ilap-counter = 0;
for ^$wnum {
worker {
until $stop {
while cas($ia-lock, 0, 1) == 1 { } # lock
LEAVE $ia-lock ⚛= 0; # unlock
++$ia-counter;
}
}
worker {
until $stop {
$il-lock.lock;
LEAVE $il-lock.unlock;
++$il-counter;
}
}
worker {
until $stop {
await $ila-lock.lock;
LEAVE $ila-lock.unlock;
++$ila-counter;
}
}
worker {
until $stop {
$iap-lock.protect: { ++$iap-counter }
}
}
worker {
until $stop {
$ilp-lock.protect: { ++$ilp-counter }
}
}
worker {
until $stop {
$ilap-lock.protect: { ++$ilap-counter }
}
}
}
await @ready;
$starter.keep;
sleep TEST_SECS;
$*ERR.print: "stop\r";
$stop ⚛= 1;
await @workers;
printf $dfmt, $wnum, $ia-counter, $il-counter, $ila-counter, $iap-counter, $ilp-counter, $ilap-counter;
}
The code is designed for a VM with 50 CPU cores available. By setting that many workers per approach, I also cover a complex case of an application over-utilizing the available CPU resources.
Let’s see what it comes up with:
Wrkrs Atomic Lock Async Atomic.P Lock.P Async.P
2 918075 665498 71982 836455 489657 76854
3 890188 652154 26960 864995 486114 27864
4 838870 520518 27524 805314 454831 27535
5 773773 428055 27481 795273 460203 28324
6 726485 595197 22926 729501 422224 23352
7 728120 377035 19213 659614 403106 19285
8 629074 270232 16472 644671 366823 17020
9 674701 473986 10063 590326 258306 9775
10 536481 446204 8513 474136 292242 7984
11 606643 242842 6362 450031 324993 7098
12 501309 224378 9150 468906 251205 8377
13 446031 145927 7370 491844 277977 8089
14 444665 181033 9241 412468 218475 10332
15 410456 169641 10967 393594 247976 10008
16 406301 206980 9504 389292 250340 10301
17 381023 186901 8748 381707 250569 8113
18 403485 150345 6011 424671 234118 6879
19 372433 127482 8251 311399 253627 7280
20 343862 139383 5196 301621 192184 5412
21 350132 132489 6751 315653 201810 6165
22 287302 188378 7317 244079 226062 6159
23 326460 183097 6924 290294 158450 6270
24 256724 128700 2623 294105 143476 3101
25 254587 83739 1808 309929 164739 1878
26 235215 245942 2228 211904 210358 1618
27 263130 112510 1701 232590 162413 2628
28 244143 228978 51 292340 161485 54
29 235120 104492 2761 245573 148261 3117
30 222840 116766 4035 241322 140127 3515
31 261837 91613 7340 221193 145555 6209
32 206170 85345 5786 278407 99747 5445
33 240815 109631 2307 242664 128062 2796
34 196083 144639 868 182816 210769 664
35 198096 142727 5128 225467 113573 4991
36 186880 225368 1979 232178 179265 1643
37 212517 110564 72 249483 157721 53
38 158757 87834 463 158768 141681 523
39 134292 61481 79 164560 104768 70
40 210495 120967 42 193469 141113 55
41 174969 118752 98 206225 160189 2094
42 157983 140766 927 127003 126041 1037
43 174095 129580 61 199023 91215 42
44 251304 185317 79 187853 90355 86
45 216065 96315 69 161697 134644 104
46 135407 67411 422 128414 110701 577
47 128418 73384 78 94186 95202 53
48 113268 81380 78 112763 113826 104
49 118124 73261 279 113389 90339 78
50 121476 85438 308 82896 54521 510
Without deep analysis, I can make a few conclusions:
Lock
. Sometimes it is even indecently faster, though
these numbers are fluctuations. But on the average it is ~1.7 times as fast as
Lock
.Lock.protect
is actually faster than Lock.lock
/LEAVE Lock.unlock
. Though
counter-intuitive, this outcome has a good explanation stemming from the
implementation details of the class. But the point is clear: use the protect
method whenever applicable.Lock::Async
is not simply much slower, than the other two. It demonstrates
just unacceptable results under heavy loads. Aside of that, it also becomes
quite erratic under the conditions. Though this doesn’t mean it is to be
unconditionally avoided, but its use must be carefully justified.And to conclude with, the performance win of atomic approach doesn’t make it a clear winner due to it’s high CPU cost. I would say that it is a good candidate to consider when there is need to protect small, short-acting operations. Especially in performance-sensitive locations. And even then there are restricting conditions to be fulfilled:
In other words, the way we utilize CPU matters. If aggregated CPU time consumed
by locking loops is larger than that needed for Lock
to release+acquire the
involved cores then atomic becomes a waste of resources.
By this moment I look at the above and wonder: are there any use for the atomic approach at all? Hm… 😉
By carefully considering this dilemma I would preliminary put it this way: I would be acceptable for an application as it knows the conditions it would be operated in and this makes it possible to estimate the outcomes.
But it is most certainly no go for a library/module which has no idea where and how would it be used.
It is much easier to formulate the rule of thumb for Lock::Async
acceptance:
Sounds like some heavily parallelized I/O to me, for example. In such cases it
is less important to be really fast but it does matter not to hit the
max_threads
limit.
This section would probably stay here for a while, until Ukraine wins the war. Until then, please, check out this page!
I have already received some donations on my PayPal. Not sure if I’m permitted to publish the names here. But I do appreciate your help a lot! In all my sincerity: Thank you!
]]>