Designing Code for Testing: An Example Implemented in Python

Recently someone asked me the following question about mocking in Python, and my answer touched on several aspects of designing code for testing. So I decided to turn it into a post.

Q and A

Question:

I am new to testing and need to write a test in [python].

I have a method I need to test:

import pysftp

cnopts = pysftp.CnOpts()
cnopts.hostkeys = None

def do_things(ids):

# iterate over ids to gather data
for id in ids:
  things = []
  for foo in thing.checkid(id):
    things.extend(#getsomestuff)

# create and write a file

# transfer that file via sftp
   try:
    with pysftp.Connection(host, username=username, password=password, cnopts=cnopts) as sftp:
        sftp.put(csv_path)
        sftp.close()
        # delete some values from the db

   except Exception as e:
      logging.log('Error connecting', exception=e)
      return ''

How do I test this without:

  1. actually creating and sending a file
  2. without deleting values from the database

I have seen something like:

@mock.patch(pysftp)
def test_do_things()
  # get some response? 
  # avoid actually connecting via sftp?
  # avoid the database change that happens after the file transfer? 
  # work with some fake data that won't impact real data?

Answer:

So there are a couple of different approaches to mocking. It looks like you’ve researched at least one annotation that helps you mock in Python. Sweet! Before we do that, there are some more fundamental steps we can take to help make testing easier, and to replace the functionality we don’t want to test when we’re in a test method.

You want to test this method, whose function (I’m inferring from reading the code) is to

  1. iterate over a list of things,
  2. map them to some other things,
  3. create a file
  4. write the other things into the file
  5. put that file somewhere via sftp
  6. delete the things from the database.

Does that sound correct?

But you want to test this without performing steps 3, 5, or 6.

So, by deduction, I’m getting that you do want to test steps 1, 2, and 4. Is that correct?

I’d also argue for testing step 3. You might want to make sure that file is created, has the data, and has the right filename (especially if some crawler is going to pick up your file at the other end of the drop location based on its name). But maybe you don’t want to because it’s too big or for some other reasons that I don’t know. Either way, the process I’m about to recommend is largely the same.

So, let’s separate your steps into separate methods. Perhaps you have:

a) a method that does steps 1 and 2
b) a method that does step 3 (and maybe 4)
c) a method that does step 5
d) a method that does step 6

Now we have separated the logic we want to perform in tests and the logic we do not want to perform in tests. Sweet.

import pysftp

cnopts = pysftp.CnOpts()
cnopts.hostkeys = None

def do_things(ids, 
              step1=self.map_stuff_from, 
              step2=self.create_and_write_file_with,
              step3=self.sftp_transfer,
              step4=self.delete_stuff_from
              ):
    stuff_from_ids = step1.call(ids)
    file = step2.call(stuff_from_ids)

    success, error = step3.call(file)
    step4.call(ids) if success else logging.log('Error connecting', exception=error)

def map_stuff_from(ids):
    for id in ids:
        things = []
        for foo in thing.checkid(id):
            things.extend(#getsomestuff)
    return things

def create_and_write_file_with(stuff_from_ids):
    #create and write file
    #return file

def sftp_transfer(file):
    try:
        with pysftp.Connection(host, username=username, password=password, cnopts=cnopts) as sftp:
            sftp.put(csv_path)
            sftp.close()
            # delete some values from the db
        except Exception as e:
            return False, e.message

    return true, '' #empty string instead of None so that clients can shove this value into a string without checking for None

def delete_stuff_from(ids):
    #do the deleting

So now our method do_stuff accepts some additional parameters. We’re passing in functions (yay for functions as first class objects!) that do the steps we want to do. But we don’t want to perform all of these functions as-is in our tests. So in our tests, we can make alternative functions for those steps we don’t want to perform and pass them in during testing.

def test_do_things():
    test_ids = [12324,353453,34242]
    
    do_things(test_ids,
       step1=map_stuff_from_ids,
       step2=create_and_write_file_with,
       step3=standin_stfp_method_that_does_nothing,
       step4=standin_delete_method_that_does_nothing)
    
    #assert some stuff in the test
    
def standin_sftp_method_that_does_nothing(file):
    return True, ''

def standin_delete_method_that_does_nothing(ids):
    pass #For evaluation purposes only

or, if you’d like more granular assertions, you can test each of your step methods individually:

def test_map_stuff_from_ids():
    test_ids = [12324,353453,34242]

    assert map_stuff_from_ids(test_ids)
       == [Object1, Object2, Object3]

def test_create_and_write_file():
    stuff_from_ids = [Object1, Object2, Object3]
    file = create_and_write_file_with(stuff_from_ids)
    
    assert file.name == 'AwesomeFileName.json'
    
#...and so on

Separating the logic you want to test from the logic you don’t want to test gives you the opportunity to introduce stand-ins in only those places where you need to avoid side-effects, while executing and asserting on everything else.

In this particular case, it also gives your code some modularity and legibility that’s harder to achieve with a large method that does many things.

Chelsea, why did you treat functions as first class objects instead of creating classes?

The code in the question does not appear to live inside an object. In python, ruby, and other scripting languages, you can do this.

I have design justifications for putting code in classes instead, but that wasn’t the question. I wanted to answer the question without introducing the additional component of class-based software design. That said, we are now in a blog post and not the question, so we will go a step further and discuss that architecture now :).

An object-oriented approach would identify separate responsibilities in this code and create a class to handle each of those responsibilities. For example, modifying files, transferring files, and making changes to the database all sound to me like separate responsibilities, each communicating with separate collaborators (the files, an sftp drop point, and the database, respectively).

the methods that contained behavior meant to be tested and the methods that contained behavior not to be tested into two different classes, with a third class that combined them.

So it might look something like this:

import pysftp

class FileProcessor():

     def __init__(self, file_manager, sftp_manager, database_manager): 
         self.file_manager = file_manager
         self.sftp_manager = sftp_manager
         self.database_manager = database_manager

     def do_things(self, ids): 
         stuff_from_ids = self.map_stuff_from(ids)
         file = file_manager.create_and_write_file_with(stuff_from_ids)

         success, error = sftp_manager.sftp_transfer(file)
         database_manager.delete_stuff_from(ids) if success else logging.log('Error connecting', exception=error)

     def map_stuff_from(self, ids):
         for id in ids:
             things = []
             for foo in thing.checkid(id):
                 things.extend(#getsomestuff)
         return things

class FileManager():

     def create_and_write_file_with(self, sstuff_from_ids):
         #create and write file
         #return file

class SftpManager():

     def __init__(self): 
         self.cnopts = pysftp.CnOpts()
         self.cnopts.hostkeys = None

     def sftp_transfer(self, file):
         try:
             with pysftp.Connection(host, username=username, password=password, cnopts=cnopts) as sftp:
                 sftp.put(csv_path)
                 sftp.close()
                 # delete some values from the db
             except Exception as e:
                 return False, e.message

         return true, '' #empty string instead of None so that clients can shove this value into a string without checking for None
 
class DatabaseManager():

     def delete_stuff_from(self, ids):
         #do the deleting

I am not intimately familiar enough with the functionality we’re talking about to definitively say that these are the correct class delineations for this code. However, this exemplifies dividing methods into classes so that each class has one responsibility. For each of these, then, a test double could be rolled like this:

class DummyDatabaseManager(DatabaseManager):

     def delete_stuff_from(self, ids):
         pass

and then injected at test time like so:

def test_do_things()
  dummy_database_manager = DummyDatabaseManager() 
  dummy_sftp_manager = DummySftpManager()
  dummy_file_manager = DummyFileManager()
  subject_under_test = FileProcessor(dummy_file_manager, dummy_sftp_manager, dummy_database_manager)
  subject_under_test.do_things([1,2,3])
  # assert some stuff

Follow-Up Question

In addition to the above discussion, the asker had a follow-up question specifically about testing the SFTP connection. I’ll also share that here because it allows us to cover some of the practical considerations of whether and how to test.

Question:

I have a method:

def _transfer_file(self, my_file):
            try:
                    with pysftp.Connection('host', username=username, password=password) as sftp:
                    sftp.put(my_file)
                    sftp.close()
                    return True

            except Exception as e:
                    # log an error with exception
                    return None

My issue is how do I test that specific method? I think it would be sufficient to simple verify that the connection can be successfully opened but how do I do this? My test would end up sending “my_file” which I do not really want to do.

I suspect I will need to use @mock.patch(something) to fake a connection? I’m not really sure. Does anyone have any suggestions on how to test this situation?

The intention behind testing the functionality would be to make sure that the API is doing what you expect. But it sounds like your test for the sftp stuff asserts on the return True value.

If you removed all the lines of code between try and return True, such a test would pass.

So it’s not asserting anything about the functionality (or even the presence) of the sftp code.

You’re calling on an external library for the sftp transfer, so if you really, really want to test this, you have two options:

  1. A unit level test: write a wrapper class that extends pysftp.Connection, overwrite the methods putand close to return values that indicate those methods were called, add your instance of pysftp.Connection to your method arguments, inject an instance of your wrapper class into this method during testing, and assert on the bogus return values from your overwrites. This ensures that the correct methods were called on the library (still not asserting on the sftp functionality itself).
  2. An integration level test: Write an overarching test with access to both your code and the sftp drop directory. Run your code and then assert that the directory contains the file you meant to transfer.

I would time box my effort to 90 minutes and spring for option #2. If I was getting nowhere with option #2 after 90 minutes, I’d leave a comment warning that this code is not tested and linking to the docs for the library.

I’m not likely to bother with option #1. It’s tightly coupled to the API of the dependency, rather than the functionality itself. I recommend using option #1 in only one circumstance: the circumstance where your boss is evaluating you on % code coverage. This method will cover your lines—but that’s about all it does.

I scanned the docs for pysftp.Connection. It seems well-documented and well-used, so the maintainers have some community pressure to keep it working. This works in your favor for sure. But just in case, like I said, I’d leave a comment. So if this code starts mysteriously not working even though your tests pass, another programmer will have a helpful clue as to where to look for potential issues.

If you found this example helpful, you might also like:

Performance Optimization: An Example Implemented in Ruby

The Numpy API: Resources, Idiosyncracies, Examples

Spring Batch with Java Config: An Example

One comment

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.