steve [ARCHIVE] on Nostr: 📅 Original date posted:2012-09-26 📝 Original message:-----BEGIN PGP SIGNED ...
📅 Original date posted:2012-09-26
📝 Original message:-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi Matt,
Glad to have another ninja onboard :)
On 25/09/2012 21:41, Matt Corallo wrote:
> Although Jenkins may not be the best system, we already have
> jenkins and pull-tester (which is a dumb python script I wrote to
> test all incoming pull requests from github).
I have never heard of jenkins before. I need to do some more digging.
is this the right thing?
https://wiki.jenkins-ci.org/display/JENKINS/Mantis+Plugin
Mantis on the other hand, I know exceptionally well. I hate
duplication of work/data unless absolutely necessary. I will check
jenkins out (just out of interest what is it actually meant to do? the
website implies framework, but not what its for)
So, currently there are 4 potential places for bugs to be reported
1 - jenkins (and unit tests)
2 - git
3 - mailing list
4 - forum (bitcointalk...)
5? - is there still the ability to add bugs via sourceforge?
Adding to this doesnt make sense. Each one of these reporting methods
is for a different thing. I am not seeking to replace these (or even
unify them) I am looking for software that will take testcases and bug
reports against them [and allow for test campaigns]. Mantis is so
flexible and industry standard and if the jenkins plugin works... then
we can keep things as they are until they fit into better places.
The reason I am so behind mantis as the backbone is it works with more
or less anything, and can easily modded to work with whatever people
are most comfortable with - however it is exceptionally powerful and
has had a constant stream of workflow improvements over the past few
years.
>
> They both run the same set of scripts, namely those at
> https://github.com/TheBlueMatt/test-scripts (its pretty basic right
> now, but since it is on github, I was hoping someone would find
> the inspiration to add to it).
I will check it out. I wrote a very basic script that wikified the
changelog, and linked to the changes and created wiki pages for the
testcases. have you seen the stuff I put on bettermeans? bits keep
vanishing then re appearing.
This is the outline of the testing that I setup for 0.7
https://secure.bettermeans.com/projects/4256/wiki
>
> I dont really care if we keep using jenkins, but I figure we might
> as well keep all the tests in one place?
Yes, I would love to unify all build testing and testcases into one
place. I am still on the fence as to including unit tests into this.
However I do see 3 distinct type of testcases
1 - requirements based testcases (requirements based off the current
block chain rules - these are edge cases and known interoperability
issues)
2 - Acceptance based testcases - these are testcases that should be
run for every build. Check out the General Acceptance Tests in the
wiki link for examples and testcases
3 - Testcases for reference implementations of things (like multisig -
i see these working like the /test folder when you install a new perl
module)
These three things alone are a massive task. and they still wont cover
everything. I would like to get the workflow so that people can
sponsor or donate to a specific campaign (eg a new feature is
implemented, people want it tested so can donate just for that
campaign [developing testcases, structure, requirements, etc])
Once this is done, I will get to do some exciting stuff (like writing
fuzzers, automation, etc) unfortunately I do not know python, only perl.
>
> Anyway, I'm all for more testing (I'm always complaining about how
> we need more tests for stuff...).
Nice, I love testing. I think we will get on :)
And I would rather go for interoperability between testing rather than
rewriting it all.
Cheers,
steve
>
> Matt
>
> On Tue, 2012-09-25 at 19:32 +0100, steve wrote:
>> Hi All,
>>
>> After the failure to get any real testing done for the 0.7
>> release (all of which is my fault) I have decided to rejig
>> things.
>>
>> I am heavily into test driven development, and I have a strong
>> background in requirements management, and automation.
>>
>> I want to leave bettermeans behind, maybe we might be able to
>> keep the voting aspect of it, and link it into mantis.
>>
>> So, what I have been doing over the past few weeks is developing
>> a rudimentary requirements set, basic requirement tracking, tests
>> to prove/stress the requirements.
>>
>> The next most important thing is to get release/acceptance tests
>> done - these primarily focus on new stuff doesnt break old (ie
>> lose a wallet, etc) and needs no special requirements.
>>
>> To this end I have installed various opensource applications
>> (mantis, salomeTMF, bugzilla, etc) and am currently evaluating
>> the best workflow process.
>>
>> This was a much longer post, but decided against it. :)
>>
>> So, what I want to know is who wants to be a part helping me out
>> with all this? I am finalising the workflow flow diagrams and
>> they should be ready for inspection soon.
>>
>> Anyone interested in helping out/reviewing processes? even if it
>> is just some encouragement, it is all greatly appreciated.
>>
>> Drop me an email if you want access to the current setup and help
>> me review the different software for the bitcoin workflow
>> process.
>>
>> cheers,
>>
>> steve
>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (MingW32)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/
iQEcBAEBAgAGBQJQYvT4AAoJEFvEB9dQFvtQlgkIAJX7JYel5RGmCsbptGdQrCnT
BR42tUwTg1t/NRUJ6RA8/Ou8lzallztQquShpLn4mZdQpoalvETdtAwcPnQKnaZb
M5inZE/IEq8WJM1y4YkHt3BLou4BJbjwncCNy1/jqcm6f2Oonrg7isVbDwY/7JlP
y/epm7XELS7NU4vVubBwQCunwvtsuydXRzuI812LiLXNqpXFMHvG2m8a2RajXE0/
xW4lOMy/hUFzEgYRQWCTAru4Ts2x3Xt26NaEUh/uKvHLwBZJ4xbdu3gpupiPb4sI
bCHnVFOC7zoQKOAnfPkCMyvtyoqpzM9HW2+DWI51FoOz851Y2F36N3Fpk/2lii4=
=W5xI
-----END PGP SIGNATURE-----
📝 Original message:-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi Matt,
Glad to have another ninja onboard :)
On 25/09/2012 21:41, Matt Corallo wrote:
> Although Jenkins may not be the best system, we already have
> jenkins and pull-tester (which is a dumb python script I wrote to
> test all incoming pull requests from github).
I have never heard of jenkins before. I need to do some more digging.
is this the right thing?
https://wiki.jenkins-ci.org/display/JENKINS/Mantis+Plugin
Mantis on the other hand, I know exceptionally well. I hate
duplication of work/data unless absolutely necessary. I will check
jenkins out (just out of interest what is it actually meant to do? the
website implies framework, but not what its for)
So, currently there are 4 potential places for bugs to be reported
1 - jenkins (and unit tests)
2 - git
3 - mailing list
4 - forum (bitcointalk...)
5? - is there still the ability to add bugs via sourceforge?
Adding to this doesnt make sense. Each one of these reporting methods
is for a different thing. I am not seeking to replace these (or even
unify them) I am looking for software that will take testcases and bug
reports against them [and allow for test campaigns]. Mantis is so
flexible and industry standard and if the jenkins plugin works... then
we can keep things as they are until they fit into better places.
The reason I am so behind mantis as the backbone is it works with more
or less anything, and can easily modded to work with whatever people
are most comfortable with - however it is exceptionally powerful and
has had a constant stream of workflow improvements over the past few
years.
>
> They both run the same set of scripts, namely those at
> https://github.com/TheBlueMatt/test-scripts (its pretty basic right
> now, but since it is on github, I was hoping someone would find
> the inspiration to add to it).
I will check it out. I wrote a very basic script that wikified the
changelog, and linked to the changes and created wiki pages for the
testcases. have you seen the stuff I put on bettermeans? bits keep
vanishing then re appearing.
This is the outline of the testing that I setup for 0.7
https://secure.bettermeans.com/projects/4256/wiki
>
> I dont really care if we keep using jenkins, but I figure we might
> as well keep all the tests in one place?
Yes, I would love to unify all build testing and testcases into one
place. I am still on the fence as to including unit tests into this.
However I do see 3 distinct type of testcases
1 - requirements based testcases (requirements based off the current
block chain rules - these are edge cases and known interoperability
issues)
2 - Acceptance based testcases - these are testcases that should be
run for every build. Check out the General Acceptance Tests in the
wiki link for examples and testcases
3 - Testcases for reference implementations of things (like multisig -
i see these working like the /test folder when you install a new perl
module)
These three things alone are a massive task. and they still wont cover
everything. I would like to get the workflow so that people can
sponsor or donate to a specific campaign (eg a new feature is
implemented, people want it tested so can donate just for that
campaign [developing testcases, structure, requirements, etc])
Once this is done, I will get to do some exciting stuff (like writing
fuzzers, automation, etc) unfortunately I do not know python, only perl.
>
> Anyway, I'm all for more testing (I'm always complaining about how
> we need more tests for stuff...).
Nice, I love testing. I think we will get on :)
And I would rather go for interoperability between testing rather than
rewriting it all.
Cheers,
steve
>
> Matt
>
> On Tue, 2012-09-25 at 19:32 +0100, steve wrote:
>> Hi All,
>>
>> After the failure to get any real testing done for the 0.7
>> release (all of which is my fault) I have decided to rejig
>> things.
>>
>> I am heavily into test driven development, and I have a strong
>> background in requirements management, and automation.
>>
>> I want to leave bettermeans behind, maybe we might be able to
>> keep the voting aspect of it, and link it into mantis.
>>
>> So, what I have been doing over the past few weeks is developing
>> a rudimentary requirements set, basic requirement tracking, tests
>> to prove/stress the requirements.
>>
>> The next most important thing is to get release/acceptance tests
>> done - these primarily focus on new stuff doesnt break old (ie
>> lose a wallet, etc) and needs no special requirements.
>>
>> To this end I have installed various opensource applications
>> (mantis, salomeTMF, bugzilla, etc) and am currently evaluating
>> the best workflow process.
>>
>> This was a much longer post, but decided against it. :)
>>
>> So, what I want to know is who wants to be a part helping me out
>> with all this? I am finalising the workflow flow diagrams and
>> they should be ready for inspection soon.
>>
>> Anyone interested in helping out/reviewing processes? even if it
>> is just some encouragement, it is all greatly appreciated.
>>
>> Drop me an email if you want access to the current setup and help
>> me review the different software for the bitcoin workflow
>> process.
>>
>> cheers,
>>
>> steve
>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (MingW32)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/
iQEcBAEBAgAGBQJQYvT4AAoJEFvEB9dQFvtQlgkIAJX7JYel5RGmCsbptGdQrCnT
BR42tUwTg1t/NRUJ6RA8/Ou8lzallztQquShpLn4mZdQpoalvETdtAwcPnQKnaZb
M5inZE/IEq8WJM1y4YkHt3BLou4BJbjwncCNy1/jqcm6f2Oonrg7isVbDwY/7JlP
y/epm7XELS7NU4vVubBwQCunwvtsuydXRzuI812LiLXNqpXFMHvG2m8a2RajXE0/
xW4lOMy/hUFzEgYRQWCTAru4Ts2x3Xt26NaEUh/uKvHLwBZJ4xbdu3gpupiPb4sI
bCHnVFOC7zoQKOAnfPkCMyvtyoqpzM9HW2+DWI51FoOz851Y2F36N3Fpk/2lii4=
=W5xI
-----END PGP SIGNATURE-----