TWIMC: tests using random and current time

[juan_gandhi]

 So, if you think you call a function in your code, and this function returns current time, or a random number... IT'S NOT A FUNCTION. Your code is function of "random number", or "time".

So, if your code is written as something that retrieves this kind of data, to test your code, you should provide that data. Not just today, but try the time, like 10 years from now. As to "random", You provide the randomness. If your code cannot be fixed to behave as a function of those inputs, make your "random stream" or "time stream" not hard-coded, but substitutable. Mockable. And mock it in your tests. MAKE SURE that you don't provide just happy-path data. Provide anything. A sequence of 100 numbers 4 for random. Time that is 10 years from now. Or even 30 yeas from now.

Make sure that your tests don't depend on anything. Because test Must Be Reproducible.

All these things, I know, are obvious to some, and not obvious to others.

If you still have questions, ask. But don't argue. Because what I say is math. Unless you have another math (some people do), or another logic (there's plenty of them), please don't argue.

I'd be glad to see how all this changes if logic is e.g. linear. 

 

Comments

[perdakot]

Диск и сеть туда же.

[juan_gandhi]

Вот; и это тоже.

[a2is]

Интерактивные тесты - туда же :)

[mikkim08]

У меня было такое задание на интервью. Написать довольно простой вебсервис, который зависит от времени, и юнит-тест для него.

[juan_gandhi]

Ну это как-то довольно свирепо уже.

[mikkim08]

Да нет, мне в целом интервью понравилось :)) Мне дали лаптоп, чтобы на нём программировать, интервьюер сидел рядом и очень доброжелательно помогал. Типа, pair-programming.

Сначала мы быстро-быстро написали простенький сервис (ипользуя jetty), а потом писали тесты, для чего нужно было рефакторить код. При этом пришлось выковыривать из бизнес-логики системный таймер и явно передавать его извне. А потом для тестов этот таймер "замочить" (от слова mock).

Ушло на это всё немногим больше часа.

What to test in time-dependent function?

[dennisgorelik]

> потом писали тесты, для чего нужно было рефакторить код

Что именно вы тестировали, что требовало рефакторинг?

[spamsink]

Or even 30 yeas from now.

Remember, remember the 19th of January 2038.

[juan_gandhi]

Exactly.

[sassa_nf]

> A sequence of 100 numbers 4 for random.

I am not sure I understand what one could do about it. How do you defend against this? Write a piece of code for validating the randoms provided are sufficiently random?

It makes more sense to write tests supposing the entropy is depleted. (and the rng device blocks indefinitely)

[juan_gandhi]

This one is basically a joke. But if people count on dispersion...

Regarding blocks indefinitely, we would be testing the RNG itself here, right?

[sassa_nf]

I think I get your point now. We shouldn't mock the entire RNG, but we do need to keep track what seed was used for a given test, so you can re-run the test, if it is seen to fail.

Re: blocking indefinitely - I was thinking of /dev/random. We can deplete it, and we have seen what happens to the programs that don't think that's possible.

[norian]

рандомность в тестах часто обрабатывается записью какой-то достаточно длинной (для создания субъективной иллюзии) последовательности и канонического, прочеканого лапками, вывода, ну то есть регрешн

время есть область ответственности оси, прикладной софт не должно парить что где там переполнилось вообще

[juan_gandhi]

По первому вопросу - это мы happy path проходим? Его, конечно, тоже надо. Но вообще речь была лишь о воспроизводимости.

[norian]

это когда пишецца простой логгер, который вкрячиваецца в реальный источнег рандомной цыфири и потом его изображает, ну и конечно можно добавить туда чего угодно лапгами при желании

How to test nondeterministic functions

[dennisgorelik]

> if you think you call a function in your code, and this function returns current time, or a random number... IT'S NOT A FUNCTION.

What you describe as "FUNCTION", is usually called "deterministic function".
~~~~~~~~~~
https://www.google.com/search?q=deterministic+function
Deterministic functions always return the same result any time they are called with a specific set of input values and given the same state of the database.
~~~~~~~~~~
Why did you chose using non-standard "function" label for "deterministic function" concept?

> So, if your code is written as something that retrieves this kind of data, to test your code, you should provide that data.

Not necessarily.
It depends on what exactly we are testing.
For example, if it is sufficient for us to test that nondeterministic function output contains specific phrase that does not depend on random input -- why should we provide random data to that test?

> Make sure that your tests don't depend on anything.

Test that does not depend on anything - does not test production code.
Do you use non-standard definition of word "anything" or phrase "don't depend"?

Re: How to test nondeterministic functions

[juan_gandhi]

Oh, you found some idiots and their opinions. Congratulations.

Re: How to test nondeterministic functions

[dennisgorelik]

What "idiots" are you referring to?

Re: How to test nondeterministic functions

[juan_gandhi]

People that are clueless regarding what is function.

Re: How to test nondeterministic functions

[sassa_nf]

If you actually read what the search results are about, you would see it is about SQL specifically. It is no longer "usually". No one outside SQL community uses that term. Even the second best result - google corrected the term to "deterministic algorithm", because google knows people don't know what they are talking about, and what they really mean is algorithm.

Is "nondeterministic function" a useful concept

[dennisgorelik]

> No one outside SQL community uses that term.

Your observation is incorrect.

https://www.google.com/search?q=%22deterministic+function%22
About 451,000 results

https://www.google.com/search?q=%22deterministic+function%22+-sql
About 524,000 results

Google's search count estimates are a bit weird here, but the main point is that there are plenty of "deterministic function" results with the word "SQL" in it.

> Even the second best result - google corrected the term to "deterministic algorithm"

It looks like you forgot to add quotes around "deterministic algorithm" in your search.

Infallible teacher

[dennisgorelik]

> But don't argue.

Because you are perfect and cannot make mistakes, right?

Re: Infallible teacher

[juan_gandhi]

No. Because you need to have a certain level of expertise to argue. Go ahead, demonstrate your expertise.

Re: Infallible teacher

[dennisgorelik]

> demonstrate your expertise

I demonstrated my expertise.
Unfortunately, today you are unwilling to comprehend it.

What kind of demonstration would convince you?

[http://users.livejournal.com/sorcerer-/]

Мне не очень понятно почему оно не happy path?
Мне очевидно, что любое конечное количество моков - это happy path при бесконечном оригинале.
Ну и так же, что все, что придумает конкретный программист, - это тоже happy path, т.к. он уже это придумал, и значит предусмотрел в коде.
Я считаю, что тесты бывают только двух видов: integration - что реально видит наш юзер и regression - настоящий un-happy path из баг репортов. Все остальное - карго культ.