Chapter 6.1, 6.2

What two properties must be satisfied for an input domain to be properly partitioned? 

  1. The partition must cover the entire domain (completeness)
  2. The blocks must not overlap (disjoint)

What is an Input Domain Model (IDM)?

An input domain model (IDM) represents the input space of the system under test in an abstract way.

A test engineer describes the structure of the input domain in terms of input characteristics. The test engineer creates a partition for each characteristic. The partition is a set of blocks, each of which contains a set of values. From the perspective of that particular characteristic, all values in each block are considered equivalent.

What gives more tests, each choice coverage or pair-wise coverage?

An each case coverage requires one value from each block for each characteristic. A pair-wise coverage a value from each block for each characteristic to be combined with a value from each block for each other characteristic.

Hence, a pair-wise coverage gives more tests.

Discussion: TDD and Criteria-Based Test Design

Feature image by Roysneak (Rugby Scrum)

The book for reference in all the post is Introduction to Software Testing, 2nd Edition. 

  • What is “correctness” in agile processes?

All agile methods have an underlying assumption that instead of defining all behaviours with requirements or specifications, we demonstrate some behaviours with specific tests. The software is considered correct if it passes all particular set of tests. But to be honest, no one is sure of what the term correctness mean when applied to a computer program.

  • Do TDD tests do a good job testing the software?

Test Driven development is an agile approach (agile is a mindset not a methodology). So it’s a good tool to be responsive to change, because its focus is create a system that does something as early as possible. TDD allow us to obtain critical feedback quickly as possible. For example today at work something in the backend crashed, but it’s better that if it’s going to fail, that fail as quickly as possible. In conclusion, yes, it’s good depending your focus.

  • Can we automate our tests without TDD?

Yes we can, automation is not unique to TDD

  • Can we use TDD without automating our tests?

Yes we can, but this mean it will be manual, so it will have to spend more time and it will return the human factor. So in other words, it’s a no in disguise.

  • What four structures do we use for test criteria?
  1. Test requirement
  2. Coverage criterion
  3. Minimal test set
  4. Minimum test set
  • What usually prevents our tests from achieving 100% coverage?

As discussed later, there are infinite number of inputs and can’t be explicit enumerated. But we can divide up the input space to maximize the number of faults found per case. To be honest a 100% coverage is not even a realistic or possible goal. Secondly, some requirements can’t be satisfied and are very hard to find (as a purple M&M’s according to the book). Sometimes this is because the existence of dead code that can’t be reached.

Another reason, what does 100% even mean? Where is the criteria for saying how much worst is 99% from 100%.

  • Some organizations in industry who adopt TDD report that it succeeds very well, and others report that it fails. Based on your knowledge of TDD and any experience you have, why do you think it succeeds sometimes but not all?

Even the book knows that the main cost of Agile methods for testing (in this particular case TDD) is that a lot of things are different. So it’s not easy for established teams and companies to change their mindset just like that. Therefore, sometimes TDD fits the project, sometimes it doesn’t. It’s not the methodology but how it’s implemented.

Image credits to New Line Cinema
  • A few software organizations use test criteria and report great success. However, most organizations do not currently use test criteria. Based on your knowledge and experience, why do you think test criteria are not used more?

Based on my experience, test criteria is still used, but not as it used to be. It’s more about following what the user story specifies. The term complete testing or full coverage are poor designed because the potential inputs for most programs are unlimited.

In traditional software development, system requirement are often questionable in terms of how complete and current they are. In agile methods, they are undocumented! So what do system testers do?

Amman, P. and Offutt, J. (2016)

As the book say, there are no definite answer for test criteria.

References

Amman, P., Offutt, J. (2016) Introduction to Software Testing, 2nd Edition, Cambridge Press Chapter 4 and 5. https://cs.gmu.edu/~offutt/softwaretest/

Cover art by Peter Hoey http://www.peterhoey.com


Chapter 4

At first, this is our Calc class:

public class Calc {    
static public int add (int a, int b) {
return a + b;
}
}

It only has the add method. Hence, we made failing tests for two methods to be implemented: mult and div, for multiplication and division. 

And a failing test for the add method as well.

The input values for the addition are incorrect. Also, the mult and div message are not recognized by the class.

As we can observe, all tests failed…

testAdd failed
testDiv failed
testMult failed

Then, we changed the testAdd input values, so the test won’t fail.

Now the addition will be correct.

Next, we implemented the methods for multiplication and division in the Calc class:

Calc now has three methods

Finally, we ran the tests again and had a successfull execution.

The class recognizes all Calc methods.

Week 4 exercise 3

Retrieved from SWE 437 In Class Exercise #5 JUnit. Code available in the end of the blogpost.

Question 1

Given the 4 @Test methods shown, how many times does the @Before method execute?

Just one time, it will setup the environment for the class we want to test.

Question 2

The contract for equals() states that no exceptions may be thrown. Instead, equals() is supposed to return false if passed a null argument. Write a JUnit test that verifies this property for the EH class.

@Test public void noNPE() {
assertEquals(false, eh1.equals(null));
assertEquals(false, eh2.equals(null));
}

Question 3

Using the given EH objects, write a test that verifies that equals() returns false if the objects are, in fact, not equal.

@Test public void equalsFalse() {
assertEquals(false, eh1.equals(eh2));
assertEquals(false, eh2.equals(eh3));
}

Question 4

Using the given EH objects, write a test that verifies that equals() returns true if the objects are, in fact, equal.

@Test public void equalsTrue() {
assertEquals(true, eh1.equals(eh1));
assertEquals(true, eh2.equals(eh2));
assertEquals(true, eh3.equals(eh3));
}

Question 5

Using the given EH objects, write a test to verify that hashCode() is consistent with equals. This test should fail if hashCode() is commented out (as shown), but pass if hashCode() is implemented.

@Test
public void hashConsistent() {
assertEquals(false, eh1.hashCode() == eh2.hashCode());
assertEquals(true, eh2.hashCode() == eh2.hashCode());
assertEquals(true, eh3.hashCode() == eh1.hashCode());
}

Code repository

4 of 4 test passed

https://github.com/samosunaz/wa-integration/tree/master/activities/week4_chapter3

Week 4 Delivery

In this week we didn’t do anything related to the project, otherwise we focused more on the part of getting more involved in testing and we did choose the technologies for testing in JavaScript: Mocha and Chai.

“Mocha is a feature-rich JavaScript test framework running on Node.js and in the browser, making asynchronous testing simple and fun.”

https://mochajs.org/

“Chai is a BDD / TDD assertion library for nodeand the browser that can be delightfully paired with any javascript testing framework.”

https://www.chaijs.com/
Resultado de imagen para chaijs

On the other hand, we did meet up and did the team homework, research about testing and its techniques and best practices.

Week 4 Plan

For this week we will finish some of the tasks we were supposed to finish last time as:

  • Create Software Requirements Specification Document
  • Define Unit Tests

Last time we researched about some technologies for testing. So this week we will choose one and start working on them.

Other thing we will do is personal tasks:

  1. Kevin: Work on software requirement specification
  2. Julia: Define unit tests
  3. Samuel: Set up the tests environment for the specifications.

Every task is managed in Github project tool: https://github.com/samosunaz/wa-integration/projects/1

Week 3 Review

Send email to Abraham

We made sure that both, Abraham and Gera, the product owner, agreed to have the project for the Quality and Testing class.

Research other technologies that will interact with WhatsApp API for SugarCRM.

You can find the document for this research here

Find options for unit testing in Javascript

We found good options such as Jasmine, Mocha, JsUnit, Jest, Karma,… We’re still deciding what’s the best framework for the project.

References

Week 3 Plan

Tasks for this week:

  • Send email to Abraham
  • Research other technologies that will interact with WhatsApp API for SugarCRM.
  • Research iFrame + WhatsApp
  • Find options for unit testing in Javascript.
  • Create Software Requirements Specification Document
  • Define Unit Tests
There will be an issue on Github for each task.
We will administrate the issues and other stuff as spikes in the project. Link in here

And also a little of organization in here

And we store most of the documentation in here

Semester Plan

  • Week 1
    • Planning
  • Week 2
    • Planning
  • Week 3
    • Planning
  • Week 4
    • Research SugarCRM API documentation
    • Research for the best way to integrate WhatsApp into a webpage
  • Week 5 – Week 10
    • Development
  • Week 11 – Week 13
    • Quality and Testing
  • Week 14
    • Deployment

This plan is an estimation, it can change through the time

Week 2 Plan

Welcome to the team sugar‘s quality and testing blog. For the first week we will do the following tasks:

  • Write a general description about what the project is about.
  • Discuss in the team the structure of the blog posts and the logistics in there.
  • Understand the APIs to understand the limits of our project:

Meet the editors:

Samuel Osuna @osunaztech

Julia Paola @juliapaolao

Kevin Cabrera @kevintroko


Week 2 delivery

What did really happened this week?

  • We did get a general description of what the project is all about:
  • Decide who will edit the post every end of the week. This was decided inside the team. The schedules can be find in here
  • Team lead blog has been assigned to Kevin O. Cabrera
  • Important files will be stored in a Drive folder.
  • Git repository setup in Github : https://github.com/samosunaz/wa-integration
  • We got access to a SUGAR CRM instance to start working 🙂