Using alternate dispatcher and delegator for integration tests

[Mathieu Lirzin]

Hello,

I am trying to simplify the way integration tests are currently run.
While looking at ‘org.apache.ofbiz.testtools.ModelTestSuite’ I have
found that it is possible to define alternate dispatcher and delegator
for specific tests.

This feature is not used anywhere in the framework and introduces a ton
of complexity since we can't assume that there is only one dispatcher
and delegator shared by all the tests cases.

As a consequence I would like to know if we could agree on removing this
unused feature?

Thanks.

[Taher Alkhateeb]

An example could shed some light here perhaps? What do you want to
remove from where? How is it complex?
On Sun, Oct 21, 2018 at 5:17 PM Mathieu Lirzin

Post by Mathieu Lirzin
Hello,
I am trying to simplify the way integration tests are currently run.
While looking at ‘org.apache.ofbiz.testtools.ModelTestSuite’ I have
found that it is possible to define alternate dispatcher and delegator
for specific tests.
This feature is not used anywhere in the framework and introduces a ton
of complexity since we can't assume that there is only one dispatcher
and delegator shared by all the tests cases.
As a consequence I would like to know if we could agree on removing this
unused feature?
Thanks.
--
Mathieu Lirzin
GPG: F2A3 8D7E EB2B 6640 5761 070D 0ADE E100 9460 4D37

[Mathieu Lirzin]

Hello Taher,

Post by Taher Alkhateeb
An example could shed some light here perhaps?

What kind of examples would help you?

AIUI It would be for the people using this feature outside of the
framework to provide examples why we should keep it in.

Post by Taher Alkhateeb
What do you want to remove from where?

Here is what I precisely want to remove.

[Taher Alkhateeb]

Ahh, Now I understand what you mean by looking at your patch. I
recommend next time that you copy-paste into the email thread because
people who try to access this thread using the ML archives might not
see your attachments.

At a first glance, although this feature is not used in the current
OFBiz code base. it _might_ be used by some users who are running
certain tests against specific tenants for example. So although I
would probably lean towards removing it, I'd recommend making sure
that not many people depend on this feature. Perhaps you can check in
the user ML or wait for others to share their opinions.
On Sun, Oct 21, 2018 at 10:51 PM Mathieu Lirzin

Post by Mathieu Lirzin
Hello Taher,

Post by Taher Alkhateeb
An example could shed some light here perhaps?

What kind of examples would help you?
AIUI It would be for the people using this feature outside of the
framework to provide examples why we should keep it in.

Post by Taher Alkhateeb
What do you want to remove from where?

Here is what I precisely want to remove.

Post by Taher Alkhateeb
How is it complex?

Basically each time an option is provided it adds complexity. In this
particular case, perhaps a comparaison between the two models of test
1. Launch OFBiz
2. Fetch all the tests from the components
3. Keep track which delegator/dispatcher correspond to each test suite
4. Run each test by setting its corresponding delegator/dispatcher
1. Launch OFBiz with the test delegator/dispatcher
2. Fetch all the tests from the components
3. Run the tests.
Does it help?
--
Mathieu Lirzin
GPG: F2A3 8D7E EB2B 6640 5761 070D 0ADE E100 9460 4D37

[Scott Gray]

The use of separate individual dispatchers/delegators per test suite was
introduced at the same time as the delegator rollback/reset code, and was
intended to allow for test suites to be run (and data to be reset) in
parallel.

The test framework never sees much love though so that never happened. It's
difficult to accomplish because many test suites rely on the same
underlying demo data.

I don't mind either way, just thought the history lesson might be helpful.

https://issues.apache.org/jira/browse/OFBIZ-2259
https://markmail.org/thread/zjghmk25bllthxme

Regards
Scott

Post by Taher Alkhateeb
Ahh, Now I understand what you mean by looking at your patch. I
recommend next time that you copy-paste into the email thread because
people who try to access this thread using the ML archives might not
see your attachments.
At a first glance, although this feature is not used in the current
OFBiz code base. it _might_ be used by some users who are running
certain tests against specific tenants for example. So although I
would probably lean towards removing it, I'd recommend making sure
that not many people depend on this feature. Perhaps you can check in
the user ML or wait for others to share their opinions.
On Sun, Oct 21, 2018 at 10:51 PM Mathieu Lirzin

Post by Mathieu Lirzin
Hello Taher,

Post by Taher Alkhateeb
An example could shed some light here perhaps?

What kind of examples would help you?
AIUI It would be for the people using this feature outside of the
framework to provide examples why we should keep it in.

Post by Taher Alkhateeb
What do you want to remove from where?

Here is what I precisely want to remove.

Post by Taher Alkhateeb
How is it complex?

Basically each time an option is provided it adds complexity. In this
particular case, perhaps a comparaison between the two models of test
1. Launch OFBiz
2. Fetch all the tests from the components
3. Keep track which delegator/dispatcher correspond to each test suite
4. Run each test by setting its corresponding delegator/dispatcher
1. Launch OFBiz with the test delegator/dispatcher
2. Fetch all the tests from the components
3. Run the tests.
Does it help?
--
Mathieu Lirzin
GPG: F2A3 8D7E EB2B 6640 5761 070D 0ADE E100 9460 4D37

[Taher Alkhateeb]

I keep forgetting this project carries a lot of history with it. Maybe
it's because I'm a relatively new. Thank you for sharing Scott, quite
informative.

If the main advantage is parallelization but that was not really used,
then maybe it's not super necessary. Maybe one way to think about this
is instead of making our tests faster, let's make them lighter. For
example we can switch out from integration tests to unit tests and do
away with the extra layer of resources being consumed.

Anyway, all options on the table, like Scott no firm opinion here.

Post by Scott Gray
The use of separate individual dispatchers/delegators per test suite was
introduced at the same time as the delegator rollback/reset code, and was
intended to allow for test suites to be run (and data to be reset) in
parallel.
The test framework never sees much love though so that never happened. It's
difficult to accomplish because many test suites rely on the same
underlying demo data.
I don't mind either way, just thought the history lesson might be helpful.
https://issues.apache.org/jira/browse/OFBIZ-2259
https://markmail.org/thread/zjghmk25bllthxme
Regards
Scott

Post by Taher Alkhateeb
Ahh, Now I understand what you mean by looking at your patch. I
recommend next time that you copy-paste into the email thread because
people who try to access this thread using the ML archives might not
see your attachments.
At a first glance, although this feature is not used in the current
OFBiz code base. it _might_ be used by some users who are running
certain tests against specific tenants for example. So although I
would probably lean towards removing it, I'd recommend making sure
that not many people depend on this feature. Perhaps you can check in
the user ML or wait for others to share their opinions.
On Sun, Oct 21, 2018 at 10:51 PM Mathieu Lirzin

Post by Mathieu Lirzin
Hello Taher,

Post by Taher Alkhateeb
An example could shed some light here perhaps?

What kind of examples would help you?
AIUI It would be for the people using this feature outside of the
framework to provide examples why we should keep it in.

Post by Taher Alkhateeb
What do you want to remove from where?

Here is what I precisely want to remove.

Post by Taher Alkhateeb
How is it complex?

Basically each time an option is provided it adds complexity. In this
particular case, perhaps a comparaison between the two models of test
1. Launch OFBiz
2. Fetch all the tests from the components
3. Keep track which delegator/dispatcher correspond to each test suite
4. Run each test by setting its corresponding delegator/dispatcher
1. Launch OFBiz with the test delegator/dispatcher
2. Fetch all the tests from the components
3. Run the tests.
Does it help?
--
Mathieu Lirzin
GPG: F2A3 8D7E EB2B 6640 5761 070D 0ADE E100 9460 4D37

[Mathieu Lirzin]

Post by Taher Alkhateeb
I keep forgetting this project carries a lot of history with it. Maybe
it's because I'm a relatively new. Thank you for sharing Scott, quite
informative.

Same here.

Post by Taher Alkhateeb
If the main advantage is parallelization but that was not really used,
then maybe it's not super necessary. Maybe one way to think about this
is instead of making our tests faster, let's make them lighter. For
example we can switch out from integration tests to unit tests and do
away with the extra layer of resources being consumed.

An extra argument for prefering unit tests over integration tests is
that they are far more parallelizable since they do to not rely on
shared ressources.

In order to achieve that, more refactoring would be needed to make the
code easily unit testable, such as:

- adding interfaces to decouple implementations
- adding alternate model constructors which doesn't impose passing
an XML element
- removing abuse of inheritance and replace it with delegation
- injecting dependencies instead of embedding them

So Strong +1!