| MIC | DET | BCMP(2) | BCMP(3) | BCMP(4) | BCMP(5) | BCMP(6) | BCMP(7) | BCMP(8) | BCMP(9) | BCMP(10) |
| A | A | 1.#QNAN000e+00 | 1.#QNAN000e+00 | 1.#QNAN000e+00 | 1.#QNAN000e+00 | 1.#QNAN000e+00 | 1.#QNAN000e+00 | 1.#QNAN000e+00 | 1.#QNAN000e+00 | 1.#QNAN000e+00 |
| A | AB | 1.#QNAN000e+00 | 1.#QNAN000e+00 | 1.#QNAN000e+00 | 1.#QNAN000e+00 | 1.#QNAN000e+00 | 1.#QNAN000e+00 | 1.#QNAN000e+00 | 1.#QNAN000e+00 | 1.#QNAN000e+00 |
| A | ABC | 1.#QNAN000e+00 | 1.#QNAN000e+00 | 1.#QNAN000e+00 | 1.#QNAN000e+00 | 1.#QNAN000e+00 | 1.#QNAN000e+00 | 1.#QNAN000e+00 | 1.#QNAN000e+00 | 1.#QNAN000e+00 |
| AB | A | 3.18045025e-02 | 2.29029788e-03 | 1.56838719e-04 | 1.13043040e-05 | 1.49886719e-05 | 2.91932548e-06 | 1.44237602e-06 | 3.31814369e-07 | 6.56225168e-08 |
| AB | AB | 1.04680098e-09 | 1.11022302e-16 | 0.00000000e+00 | 0.00000000e+00 | 0.00000000e+00 | 0.00000000e+00 | 0.00000000e+00 | 0.00000000e+00 | 0.00000000e+00 |
| AB | ABC | 1.04680098e-09 | 1.11022302e-16 | 0.00000000e+00 | -2.22044605e-16 | 0.00000000e+00 | 0.00000000e+00 | 0.00000000e+00 | 0.00000000e+00 | 0.00000000e+00 |
| ABC | A | 2.08653050e-02 | 9.33679776e-04 | 7.74271172e-05 | 6.18296617e-06 | 1.49886719e-05 | 2.91932548e-06 | 1.44237602e-06 | 3.31814369e-07 | 6.56225168e-08 |
| ABC | AB | 1.04680098e-09 | 1.11022302e-16 | 0.00000000e+00 | 0.00000000e+00 | 0.00000000e+00 | 0.00000000e+00 | 0.00000000e+00 | 0.00000000e+00 | 0.00000000e+00 |
| ABC | ABC | 1.04680098e-09 | 1.11022302e-16 | 0.00000000e+00 | 0.00000000e+00 | 0.00000000e+00 | -2.22044605e-16 | 0.00000000e+00 | 0.00000000e+00 | 0.00000000e+00 |
All p-values are NaN for microsimulation scenario A because all replicates are identical, meaning that the population standard deviation is zero and the t-statistic is NaN. So that is as expected.
The following pattern occurs for both microsimulation scenarios AB and ABC: as the number of replicates increases, the difference from deterministic scenario A becomes more significant, as we might expect. But the differences from deterministic scenarios AB and ABC become more significant more quickly than that from deterministic scenario A. So this is telling us (i) that there seem to be serious mismatches between deterministic and microsimulation models for AB and for ABC, and (iii), that there is either no discernible difference between the microsimulation scenarios AB and ABC or between the deterministic scenarios AB and ABC, or both. This means i earn 0 points because i failed to distinguish scenarios AB and ABC.
But why? Unfortunately, although the log file contains all the information needed to hunt down the answer(s) to that, it is exceedingly difficult to visualise what's going on, to the point where i'm seriously considering investing time in implementing a GNUplot visualisation of the situation. GNUplot would be better than "just plotting graphs in Excel" because the situation is sufficiently complicated that a lot of tedious manual organisation would be required to do so every time, and this can quite straightforwardly be automated using GNUplot. GNUplot also has more flexibility than Excel charts (multiplots and colour choices particularly come to mind), making it possible to produce graphs of the situation that are easier to understand by inspection. The guru has to be invoked to check the output, and we want to make life as easy as possible for him...
Iteration 2.9 (Week starting 2011/05/29; velocity = 0 points/iteration)
Having adhered to strict TDD with my approximately 2 minute cycle time seems to be a bit tedious at times, but i still think it is better than the alternative of trying to take steps which are too big and force assumptions to be made which can later come back to bite. The going is slow but steady with TDD, so i'm going to keep adhering strictly to it. The projection has me coming in two weeks late, but hopefully the focus will now be on the details of the model without any further significant background tasks to delay me.
• STORY(1 point): Validate scenario ABCDE, a het model with het behavioural response.
Not touched. I earn 0 points.
• STORY(1 point): Validate scenario ABCD, a model of heterosexual consensual sex HIV transmission ("het model").
Not touched. I earn 0 points.
• STORY(1 point): Validate scenario ABC, a birth-death model with HIV presence and mortality. This is the base of subsystem testing scenarios.
Not finished. I earn 0 points. I quite quickly set up the scenario and deterministitic model requirements for running scenario ABC. But when i did so, i discovered the system could not find significant differences between scenarios AB and ABC. Mait and i had a discussion about this: he suggested i implement the False Discovery Rate alternative to Bonferroni correction, and he agreed to my suggestion to implement the ability to compare the microsimulation outputs for a given model to the deterministic outputs for all models in the validation sequence in order to discover such failures to detect difference more automatically. I subsequently spent the rest of the week working on implementing the latter, but did not finish in time to do a meaningful rerun of the scenario A, AB, ABC subsequence.
Iteration 2.8 (Week starting 2011/05/22; velocity = 1 point/iteration)
This is a repeat of the last iteration's stories. The current projection has me coming in one week late again. From now on, i will make an effort to adhere to strict TDD and see where this takes us...
• STORY(1 point): Validate scenario ABCD, a model of heterosexual consensual sex HIV transmission ("het model").
Not touched. I earn 0 points.
• STORY(1 point): Validate scenario ABC, a birth-death model with HIV presence and mortality. This is the base of subsystem testing scenarios.
Not touched. I earn 0 points.
• STORY(1 point): Validate scenario AB, a simple birth-death model.
Done. I earn 1 point. I adhered to strict TDD, and spent most of the iteration finalising the automatic mechanism for comparing the two models' outputs. However, i did have time to construct the scenario setup and the deterministic model for scenario AB. I first tried running the scenario before changing the deterministic model from the constant one which worked for scenario A, in order to see whether the automatic model comparison mechanism would notice the problem. It did. I then thought i had applied the required correction to the deterministic model and reran it. But the mechanism noticed another problem. I then thought a bit more carefully about how i had been adding to the deterministic model definition and was able to correct the second error it was pointing out. I then ran the model a third time and it was satisfied that any differences between the two models were no longer significant (at hard-coded level 0.05). So the comparison mechanism seems to be working adequately. I was doing runs with 4 replicates over 3 years and it took about 450MB and 30 minutes per run. I'm not yet at the stage where it will be necessary to get a steady state for each scenario before moving onto the next, because it should be possible for a while to get scenarios to pass simply by adding pieces to the deterministic model, and checking that they both evolve the same way from the same initial conditions. Hopefully i will have the second machine available soon so i can start running longer and more thorough tests; until then i need to interleave development with test runs because my laptop can't handle both at once. This will gradually become a more significant burden until the second machine arrives...
Iteration 2.7 (Week starting 2011/05/15; velocity = 0 points/iteration)
Having now completed the bulk of the behind-the-scenes stuff (although some bugs, misconceptions, etc may still need to be ironed out), i anticipate that most of the work will now be focused on actually inspecting the behaviours of the two models under various conditions and deciding whether they match and what to do about it whenever we decide they don't. As a result, i'm hoping i will be able to work through 3 scenarios every iteration, and doing so will have me coming in just on time for the release. Exactly how thoroughly i will be able to test these scenarios now depends on how quickly a second machine (more capable than my laptop) becomes available for me to devote to this task; i am in the process of trying to organise this.
• STORY(1 point): Validate scenario ABCD, a model of heterosexual consensual sex HIV transmission ("het model").
Not touched. I earn 0 points.
• STORY(1 point): Validate scenario ABC, a birth-death model with HIV presence and mortality. This is the base of subsystem testing scenarios.
Not touched. I earn 0 points.
• STORY(1 point): Validate scenario AB, a simple birth-death model.
Not completed. I earn 0 points. I've been having trouble converting the system from one where a scenario is run and one must visually inspect the outputs afterwards to try to decide how the two models compared into one where the system itself has a mechanism for reliably making that decision automatically. I initially tried to write this mechanism without following strict TDD (because, after all, how hard could it possibly be?!?), but found myself about half-way through the iteration getting so frustrated with being unable to debug the resulting code that i decided to start it over from scratch using strict TDD. I then made some progress towards developing it correctly, but didn't manage to finish in time to run the required scenarios using it. Maybe if i had faced the supposed tedium of developing it using strict TDD (with a too-long cycle time of 2 minutes) from the outset i could have finished in time and have something to show for this iteration. I didn't think it worthwhile to run the iteration's scenarios without this mechanism in place; doing so would just have distracted me from making progress with implementing the mechanism because (i) i can't run scenarios and develop on my laptop simultaneously for fear of overheating it, and (ii) having to interpret the results manually afterwards would have distracted me further...
Iteration 2.6 (Week starting 2011/05/08; velocity = 1 point/iteration)
I haven't yet completed the behind-the-scenes stuff. I have an initial deterministic model ready to use, and i have figured out most of the details of how to do the multi-point t-test that will be required to compare the outputs of the deterministic model and microsimulation replicates. But i have yet to fix all the wiring that will get the results of the two types of model into in a suitable form to carry out the comparison. I will stop trying to second-guess the completion-time prediction that the estimates make; they will just have to speak for themselves over time: they currently have me coming in two weeks late for this release, but i'm still fairly optimistic that at least the first three validation stories will be easy to deliver on once i've finished setting up the comparison mechanism.
• STORY(1 point): Validate scenario ABC, a birth-death model with HIV presence and mortality. This is the base of subsystem testing scenarios.
Not touched. I earn 0 points.
• STORY(1 point): Validate scenario AB, a simple birth-death model.
Not completed. I earn 0 points.
• STORY(1 point): Validate scenario A, an arbitrary fixed starting population distribution over all risk-groups, genders, HIV statuses, subject to no events whatsoever.
Done. I earn 1 point. By "done", i mean that scenario A is validatable, and i am very confident that the test would pass if it was to be run at full scale (whatever we decide that should mean). I have created the ability to (i) run the microsimulation and deterministic models for the same scenario, (ii) get the results of all microsimulation replicates into a combined TimeSeries structure where i can extract the appropriate sample means and standard deviations across replicates, (iii) use those and the deterministic population means to construct t-statistics and (iv) use a general t-table function to turn those into p-values. Since scenario A is a special case where there is no variability, the microsimulation replicates all have the same values, so their standard deviations are all zero, so the t-statistics and hence p-values are all undefined. So i can't see yet how to do the Bonferroni correction.
I did a run of scenario A with 4 replicates over 4 years and a population of size 168, and it took about 45 minutes and about 400MB to establish that the two models are exactly equivalent in this case. I repeatedly compute the sample statistics for 2, 3, ... up to all of the replicates, so that we can in principle efficiently get some feel of the impact of adding replicates to the degree of confidence we would have in the matching of model outputs, but we won't be able to draw any conclusions from that until we run the first scenario which actually has some variability (scenario AB). I had to run the aforementioned version of scenario A several times, ironing out various bugs i encountered along the way. One interesting one was discovering artifacts of using certain values for PopulationScalingFactor: if a value of PopulationScalingFactor is used which would yield a non-integral scaled value for any of the component starting population sizes, then the mechanism which creates the starting population before a replicate runs rounds the number of actors created for the subpopulation down to the greatest lower bound whole number of actors. This would sometimes cause a large relative mismatch between the initial conditions for the two models, so i have added a test which checks for this condition and warns the user when it is about to occur, but lets the run continue anyway.
Iteration 2.5 (Week starting 2011/05/01; velocity = 0 points/iteration)
I'm somewhat refreshed after my time off, so hopefully this will be the iteration where i complete the behind-the-scenes stuff and start delivering on the validation STORYs. I'm keeping the schedule and estimates as they are for now, since i haven't yet seen what's involved in delivering a validation STORY. The estimates currently have me coming in a week late for this release; hopefully i'll be able to rectify that situation in time.
• STORY(1 point): Validate scenario ABC, a birth-death model with HIV presence and mortality. This is the base of subsystem testing scenarios.
Not done. I earn 0 points.
• STORY(1 point): Validate scenario AB, a simple birth-death model.
Not done. I earn 0 points.
• STORY(1 point): Validate scenario A, an arbitrary fixed starting population distribution over all risk-groups, genders, HIV statuses, subject to no events whatsoever.
Not done. I earn 0 points.
Iteration 2.4 (Week starting 2011/04/24; velocity = 0 points/iteration)
I'm taking time off from Friday 2011/04/22 to Saturday 2011/04/30 inclusive, so this iteration won't happen.
Iteration 2.3 (Week starting 2011/04/17; velocity = 0 points/iteration)
I didn't have a good iteration here, partly because the time was cut short and i was severely underperforming due to not having had time off in 16 months, and partly because i was still busy doing stuff behind the scenes in preparation for doing these stories. What i did accomplish was to adapt an existing Runge-Kutta engine to use TimeSeries and Values; but i have not yet set up the actual deterministic model or the statistical test for comparing microsimulation and deterministic model outputs.
• STORY(1 point): Validate scenario AB, a simple birth-death model.
Not done. I earn 0 points.
• STORY(1 point): Validate scenario A, an arbitrary fixed starting population distribution over all risk-groups, genders, HIV statuses, subject to no events whatsoever.
Not done. I earn 0 points.
Iteration 2.2 (Week starting 2011/04/10; velocity = 0 points/iteration)
This story turned out to be bigger than i was able to manage in one iteration. If i want to stabilise my velocity (which would be a good thing for estimating purposes), i would need to figure out how to split stories into smaller pieces, while preserving their characteristic of providing Customer value. However, i quite like the way the stories are set out for this release and am not sure how they could be broken up into smaller pieces. I anticipate that the subsequent stories will be rather shorter once a minimalist validation infrastructure is set up (i'm guessing for now 1 point each). So i'm just carrying an estimate-reduced copy of this story over to the next iteration, and hoping that my velocity will stabilise (i'm guessing at 2.0 points/iteration) over the coming iterations because the subsequent stories are smaller.
• STORY(2 points): Validate scenario A, an arbitrary fixed starting population distribution over all risk-groups, genders, HIV statuses, subject to no events whatsoever. This STORY is really about setting up a minimalist validation infrastructure.
Not finished. I have made progress with modifying the build superstructure and the organisation of the include file components in the scenarios folder, but i have not yet added the scenarioA deterministic model implementation (or its underlying RungeKutta engine) or the statistical model comparison test to the code. I earn 0 points.
Iteration 2.1 (Week starting 2011/04/03; velocity = 2.5 points/iteration)
I completed what i set out to do for a change. (This should be the norm, by the way...)
I will be using the rest of this iteration for slack: trying to sort out performance problems, dealing with technical debt, trying to get a handle on what next week's STORY will involve, etc.
• STORY(1 point): Write RiskGroupsArchitecture, a wiki page which gives somebody studying the source code help in understanding it, without being so specifically detailed that it requires extensive maintenance as the code base changes.
Done. I've uploaded a draft to the wiki. It is supposed to be the start of an interactive discussion about the architecture, including reasons for its current nature and possible directions for its future evolution. I earn 1 point.
• STORY(0.5 points): Get the code into a condition where it can be committed as a revision worthy of the release 1 feature statement.
Done. Committed a codebase with severe performance problems, but a suitable point from which to start validating. I earn 0.5 points.
• STORY(1 point): Create an agile plan for release 2.
Done. I earn 1 point.
Release 1 (2011/03/31)
FEATURE: RiskGroups is a working microsimulation model which theoretically meets the description of the Estonian deterministic model, as the latter currently stands. It has not been tested and has performance issues which hinder testing.
Iteration 1.13 (Week starting 2011/03/27; velocity = 0.5 points/iteration)
Crunch time. The first STORY i scheduled last week proved to be considerably more complex than i had expected it to be, involving fairly significant code rewriting to work around an apparent Modgen limitation (which i am liasing with the Modgen developer to pin down). I would still like to try to complete the remaining STORYs for the FEATURE as it currently stands, but there's quite a lot still to do to achieve this. I'm trying my best...
• STORY(2 points): Use Mait's R code to implement a parallel deterministic model in C++. The most straightforward way of doing this seems to be to implement the deterministic model inside the RiskGroups Modgen code, so that it has easy access to all the same scenario parameters that the stochastic model uses. Then a parameter can be added to scenarios which dictates whether or not the deterministic model should be run.
Not touched. I don't earn any points.
• STORY(0.5 points): Instrument Stream with absolute and delta timestamps to start getting a handle on any performance bottlenecks.
Done. I earn 0.5 points.
• STORY(0.5 points): Produce log dumps of Map (post-setup and post-simulation) and StatusToPrevalenceConverter, so that their outputs can be compared.
Incomplete, so i don't earn any points.
• STORY(1 point): Liaise with Statistics Canada to isolate the true cause of the apparent limitation that Modgen has in only allowing 64 event types per actor type, and finish restructuring the event family setup around the relevant true limitations of Modgen.
Confusing, because i saw the apparent limitation, but was unable to isolate what i was convinced was the cause in a minimal test model. I have been changing the main model's code and have not seen the problem i thought i saw again, although it may still be lurking somewhere. Since this STORY is not complete, i don't earn any points for it.
Iteration 1.12 (Week starting 2011/03/20; velocity = 1.0 points/iteration)
i'm putting both stories in here because i can see i'm going to need to try at least to get started on the deterministic model implementation in this iteration, because the next iteration is only half a week long.
• STORY(2 points): Use Mait's R code to implement a parallel deterministic model in C++. The most straightforward way of doing this seems to be to implement the deterministic model inside the RiskGroups Modgen code, so that it has easy access to all the same scenario parameters that the stochastic model uses. Then a parameter can be added to scenarios which dictates whether or not the deterministic model should be run.
Not done.
• STORY(2 points): Resolve the potential issues standing in the way of the microsimulation model running to completion and producing usable results. These issues may include: (i) when running DirectedPopulationContactRateTest, why are all the input rates NaN? (ii) why does the simulation only seem to produce results for years 0 and 1 instead of the full specified range 0..20? (iii) how do the contents of Map at the end of the simulation relate to the outputs produced post-simulation by StatusToPrevalenceConverter? (iv) how can the outputs of the microsimulation model (for now, and later the deterministic model) best be produced to be usable for comparison purposes? (The choice is between writing everything to tables and outputting everything in a textual dump of Map into the log file.) (v) try to set up some sort of profiling to see if there are any obvious performance bottlenecks that can easily be resolved.
(i) The problem turned out to have been a subtle error in the way the interpolation of TimeSeries works that was masked in the way i had been testing it. Essentially, when the requested point in time is outside the range of defined values (as it was because i had not yet implemented the Time::toCalendarYear() method properly), it would assume that the requested time point had an existing Value attached to it, and would then interpolate between the actually non-existent Value at this time point (retrieved as a Value containing only NaNs) and the extreme existing Value it was closest to. The correction was simply to use the extreme Value that it was closest to instead of trying to interpolate when not between two genuinely existing Values. This error had some quite far-reaching ramifications, including causing some events to fail because NaN rates were retrieved. We now explicitly test in all event time methods that the rate returned is not NaN. This problem has been resolved.
(ii) The problem seems to be that Modgen internals somehow impose a limit of 64 different events per actor type: When the Ticker's IntroductionEvent family contained 64 members, there was evidence that all of its 64 members were being applied, but the TickEvent wasn't. But when the IntroductionEvent family was reduced in size, TickEvent was suddenly being applied. This limitation seems to require me to split the event families over collections of universal actors with names encoding the risk group indices involved in the families. This problem is in the process of being resolved.
(iii) Not done.
(iv) Not done.
(v) Not done yet, but i have investigated that VS 2008 Professional doesn't come with a profiler (that requires VS 2008 Team Suite), so i can't do statistical or instrumented profiling. However, there was a technique i used in DEBI (associating all log output with absolute and delta timestamps) that seems to be fairly straightforward to implement (in Stream) and should allow some performance problems to be uncovered by interactively manually instrumenting the code to trace where it spends the bulk of its time during shortish toy simulation runs. Not ideal, but i may have to pursue this tactic a bit if the performance of simulations becomes unbearably slow.
Iteration 1.11 (Week starting 2011/03/13; velocity = 2.0 points/iteration)
• STORY(2 points): Modify the implementation to match the new ModelTransformation document. This mostly requires ensuring (i) that the user-specifiable parameters are correct, (ii) that MapEst maps them correctly into the internal general model, and (iii) that the PrevalenceResponse class and behaviour groups are appropriately used to modify symmetric and directed contact rates in the internal model. (Part (ii) may be sufficient to deal in passing with whatever i meant by the old story "Incorporate supplied prophylactic factors into SymmetricTransmissionProbability, ReceiverInfectingTransmissionProbability, SenderInfectingTransmissionProbability at beginning of simulation.")
i have tried to follow the spirit of this story, but in the process i have encountered some issues with actually running the microsimulation model to completion and having it produce usable outputs. i am regarding this story as done, but am spawning another one to deal with these issues. concerning the letter of this story: (i) the user specifiable parameters are all present and in the correct forms, but their values are not yet correct, and we may need to resolve some misunderstandings about what these values mean before this can be fixed. i think such discussions should be deferred to the validation process in the next release. (ii) i'm fairly confident that MapEst is set up correctly, except for the commercial sex directed contact rates: i have the three possibilities for specification of contact rates, and i'm not sure i'm using them all correctly. (iii) has been done.
• STORY(0 points): Finish setting up the PrevalenceResponse usage. This should now be very quick: I just have to deal with some compiler errors and actually call the PrevalenceResponse::getBehaviouralResponse() method in the appropriate ways in the appropriate places in the DirectedContacts and SymmetricContacts modules.
done.
Iteration 1.10 (Week starting 2011/03/06; velocity = 2.0 points/iteration)
• STORY(1.5 points): Do performance testing to investigate why Map*::setupDirectedContacts() takes so long. It may be useful to reimplement Value to use a heap-allocated double array (perhaps flat with a suitable one-to-one Index to index correspondence) rather than the map it currently crawls along with when creating Value arrays with many entries. Use the array implementation if it proves significantly faster.
done.
• STORY(0.5 point): Finish ensuring that PrevalenceResponse is correctly used in the rest of the program.
almost done. a zero point STORY has been put into the next iteration to tie up a few loose ends.
Iteration 1.9 (Week starting 2011/02/27; velocity = 0.5 points/iteration)
although the last iteration demonstrated that my velocity is quite resilient to wishful thinking about the pace i can achieve, i do still need to keep aiming to improve it. i'm going for 3 points this time.
• STORY(2 points): Modify the implementation to match the new ModelTransformation document. This mostly requires ensuring (i) that the user-specifiable parameters are correct, (ii) that MapEst maps them correctly into the internal general model, and (iii) that the PrevalenceResponse class and behaviour groups are appropriately used to modify symmetric and directed contact rates in the internal model. (Part (ii) may be sufficient to deal in passing with whatever i meant by the old story "Incorporate supplied prophylactic factors into SymmetricTransmissionProbability, ReceiverInfectingTransmissionProbability, SenderInfectingTransmissionProbability at beginning of simulation.")
still not started...
• STORY(1 point): Finish implementing PrevalenceResponse and use it in the rest of the program. To finish the implementation (i) add the update mechanism with user-specifiable frequency, and (ii) find a computationally efficient way to return the behaviour-dependent response (i.e., one that doesn't put the expensive exponentiation call inside some very tight inner loop).
not finished yet. i have added the update event. i have attempted to set up a test of the computational efficiency of the behaviour-dependent response, and have as a preliminary test compared the efficiency of the simple method with one that treats the null behaviour specially. i'm not sure how to interpret the statistical significance of the results, so i figured i should put that task on the backburner rather than trying hard to implement some result caching mechanism without having any way of telling whether my efforts are bearing any meaningful fruit. instead i figured i should get on with using the PrevalenceResponse class in the rest of the program. at this point i saw a big refactoring opportunity in the Map* modules, and essentially spent the rest of the iteration turning these modules into a class hierarchy so that code common among them could be pulled up into their common base class (in Map.mpp). having this class hierarchy also allows the Map mechanism to be unit tested: at this point i discovered a major performance problem in setting up the MapId subclass (which seemed the natural choice to use for testing purposes). then my time for the iteration ran out. (note that i only commit to the repository once i have finished a STORY so that the respository never contains code that fails to compile or pass tests. so, unfortunately, you can't see what i'm talking about yet. is that a bad thing?)
Iteration 1.8 (Week starting 2011/02/20; velocity = 1.0 points/iteration)
i'm setting an ambitious target for this iteration (4 points when my velocity shows i manage about 1.5/week), but the release date looms and i need to try to jack up the pace...
• STORY(2 points): Modify the implementation to match the new ModelTransformation document. This mostly requires ensuring (i) that the user-specifiable parameters are correct, (ii) that MapEst maps them correctly into the internal general model, and (iii) that the PrevalenceResponse class and behaviour groups are appropriately used to modify symmetric and directed contact rates in the internal model. (Part (ii) may be sufficient to deal in passing with whatever i meant by the old story "Incorporate supplied prophylactic factors into SymmetricTransmissionProbability, ReceiverInfectingTransmissionProbability, SenderInfectingTransmissionProbability at beginning of simulation.")
not started.
• STORY(2 points): Use the new TimeSeries to implement the weighted-sum-of-prevalences PrevalenceResponse class. This should be relatively straightforward once TimeSeries is functioning properly. A GRH TimeSeries will represent the population states by gender, risk-group and HIV status over time. As new data comes in (at a user-specifiable frequency), this population state TimeSeries will be updated. As behavioural response requests are made, maybe separate TimeSeries with appropriate signatures will be computed to track the various prevalences over time and possibly a behaviour group-dependent behavioural response factor over time. Care needs to be taken to ensure that this very intensively used class doesn't cause unnecessary performance bottlenecks...
i made some progress on this. population sizes and prevalences (including various aggregates) are trackable. however, i encountered a bug in TimeSeries which i had trouble isolating. i went on a bit of a tangent improving the flexibility of log outputs, so that i could have better eyes to see the bug. (so there is now a Stream class which mimics the C++ standard ostream class, which for some reason i couldn't get working.) having the flexibility to output data as i wanted while tests are run helped me to isolate the bug (which turned out to have to do with a misinterpretation of how map::insert works in the case of an existing key), and then it was easy to resolve. however, i'm still not finished with the implementation of PrevalenceResponse nor have i used it yet in the rest of the program.
• STORY(0 points): Correct the few remaining incorrect uses of TimeSeries. This should take no more than an hour or two to finish.
done.
Iteration 1.7 (Week starting 2011/02/13; velocity = 1.5 points/iteration)
i've been sick for part of this iteration, but i think i've made reasonable progress anyway...
• STORY(2 points): Finish implementing the various classes associated with TimeSeries functionality and replace all current uses of TimeSeriesGRH and TimeSeriesTGRGR with the new TimeSeries.
the TimeSeries implementation is now completely Test Driven Developed, and i'm quite satisfied that all is well with it for the forseeable future. it has undergone several design revisions in the process and it now relies neither on generated code nor any new template classes. (it uses the STL vector and map container template classes, but it doesn't define any new ones, since this could apparently have had a significant impact on build cycle time). i almost completed correcting all uses of TimeSeries throughout RiskGroups, but didn't quite manage it so i don't get the full 2 points...
Iteration 1.6 (Week starting 2011/02/06; velocity = 0.5 points/iteration)
my velocity was lower for this iteration because i've been discovering and attempting to deal with technical debt. perhaps i could have been better focused on "just delivering the story", but since i had made the story up myself (so no Customer expectation was riding on it), i considered this to be more important to get right first. as always, if you feel i'm going off track somehow, let me know by editing the above plan.
• STORY(1 point): Complete implementation of the weighted-sum-of-prevalences behavioural response mechanism.
not done yet. i tried to figure out how to use TimeSeriesGRH (which was better than the TimeDependentParameter originally developed, but was still not developed using Test Driven Development) and realised i didn't understand how to test it properly to verify that its behaviour was as i expected it to be. this frustrated me so much that i decided to reimplement TimeSeries from the ground up using strict TDD. there is now a single abstract TimeSeries class whose functionality (which is primarily figuring out which stored Values to use to perform an interpolation for a given Time and constructing the appropriate interpolated Value) is independent of the exact details of the signature of Value. Value in turn will develop either into a class hierarchy or into a template class, so that values with different signatures (at least GRH and TGRGR) can be accommodated.
• STORY(2 points): Modify the implementation to match the new ModelTransformation document.
not done yet
Iteration 1.5 (Week starting 2011/01/30; velocity = 0.5 + 1 = 1.5 points/iteration)
this iteration, i've tried using the Energised Work practice, and it has helped me to put in more hours than the previous iteration, with less strain. that hasn't translated into increased velocity for this iteration, but i do think it has potential to do so, especially if i manage to sort out the other issue i'm having... i'm experiencing a problem with the TDD build cycle for RiskGroups being about an order of magnitude too long to practice TDD in the ideal way (with really tiny test or functionality increments). this means that i have to do significantly more with the result of each compile cycle, which introduces quite significant cognitive overhead and makes it much harder to avoid straying from flow either into boredom or anxiety. unfortunately it seems to be the nature of C++ that compile cycles are quite long, even if the program is made separate compilation friendly.
• STORY(0.5 points): Finish modifying and updating ModelTransformation.
done. behaviours use a separate index set from risk groups. behavioural response factor is generally defined. needle sharing has been split by gender to conform with consensual sex (so they both consistently inform how symmetric contacts should be defined).
• STORY(2 points): Implement the weighted-sum-of-prevalences behavioural response mechanism.
this has taken me longer than i expected, because i recognised the opportunity to reuse the functionality in what was TimeDependentParameter (now TimeSeries), and needed to do quite an extensive refactoring to split the class from storing multiple parameters in one instance to storing one parameter per subclass instance with one subclass per parameter signature (TimeSeriesGRH implements GENDER x RISK_GROUP x HIV_STATUS ; TimeSeriesTGRGR implements TRANSMISSION_MODE x GENDER x RISK_GROUP x GENDER x RISK_GROUP). having done this, i'm now in a position to reuse an instance of TimeSeriesGRH to store the population size data, but i have yet to do so.
• STORY(2 points): Modify the implementation to match the new ModelTransformation document.
not done yet
Iteration 1.4 (Week starting 2011/01/23; velocity = 1.5 points/iteration)
• STORY(2 points): Modify ModelTransformation LaTeX document to reflect use of separate behaviour index set and separate prophylactic use and effectiveness probabilities for all transmission modes.
this has taken me longer to do than i expected, i suspect mostly because i'm not following the Energised Work practice. i'm almost finished, and i am going to practice Energised Work from now on, in the sense of having a timebox each working day in which i can get work done, and not trying to work outside that time and sacrifice my sleep, health, wellbeing, etc., as i would otherwise be inclined to do.
• STORY(2 points): Modify the implementation to match the new ModelTransformation document.
not done yet
• STORY(2 points): Implement the weighted-sum-of-prevalences behavioural response mechanism.
not done yet
Iteration 1.3 (Week starting 2011/01/16)
No stories
wrote StringMatrix module in order to remove dependence of the generation of ModelTransformation on SciLab, so that this LaTeX document can become a first class agile object with anyone being able to modify it (should that prove useful).
Iteration 1.2 (Week starting 2011/01/09)
• STORY: As a customer I would like the prophylactic use probability (double EstC) to depend on risk-gender group and transmission mode.
after much discussion, finally decided that EstC did not need to depend on risk-gender groups and transmission mode, but that a separate behaviour index set should be used to label interaction situations.
set up the build script infrastructure to accommodate LaTeX operations in preparation for modifying the ModelTransformation document to reflect this decision.
Iteration 1.1 (Week starting 2011/01/02)
• STORY: Make scenario files in folder 'scenarios' openable. [Currently general error prompt appears when trying to open a scenario file.]
explained that files in scenario folder are pieces of Modgen scenario files, which the build script uses an include mechanism to put together in various ways to construct conventional Modgen scenario files (which it puts in the output folder).
• STORY: Make the model to pass current unit tests. / Clean the corresponding portion of code (remove empty tests?). [When running >perl build.pl {::clean} {::cleanoutput} ::modgen ::devenv ::run(FALSE,Low,EST,PopulationContactRate,0.001,0.001), some errors about unit tests are given and simulation is not run.]
explained that what was seen was in fact a use of ::run where the unit tests passed, but the simulation was not run because it was a unit-test-only run needed for a tight test-driven development cycle.
• STORY: As a customer I would like to see short descriptions about the purpose of the file in the beginning of each MPP source file.
done
| «Main Page
| • Queries? Email: Roger Mateer
| • Last Modified: 2011/08/15
| • All rights reserved © SACEMA 2011
|
| | | | | | | | | |
