Debugging Copulas and Speeding Up Simulations with AI

Table of Links

Abstract and 1 Introduction

2 Methodology

2.1 The task

2.2 The communication protocol

2.3 The copula family

3 Pair programming with ChatGPT

3.1 Warm up

3.2 The density

3.3 The estimation

3.4 The sampling

3.5 The visualization

3.6 The parallelization

4 Summary and discussion

5 Conclusion and Acknowledgments

Appendix A: The solution in Python

Appendix B: The solution in R

References

3.5 The visualization

1. The parameters θ are linearly spaced in the desired interval. This is a typical choice for many visualizations.

   
   

2. It shows perfect identity by a line, also typically considered in an ideal benchmark. This clearly demonstrates that ChatGPT at least partially understands the underlying concepts, i.e., that we are estimating the true value of some parameter.

   
   

3. Typical time-consuming trivialities like limits, labels, title and legend are also shown.

All in all, this is the type of tasks where the user can substantially benefit from a collaboration with tools like ChatGPT.

However, we would like to note that two iterations of our last prompt were done before we got the presented one. In the first one, we omitted the content of the two parentheses with the function names. In that case, the output of ClaytonSample did not match the dimension of the input of ClaytonCopulaMLE. In the second iteration, we added those parentheses to the prompt, but without “U = ”, and got a very similar error. We copy-pasted the error message to ChatGPT, but this time it was not succesful to provide a correct version. Finally, we added that “U = ” in the prompt with the intuition that ChatGPT is not aware of the right dimensions of the output, and this way we got a working solution. The main take-away from this example is that obtaining a correct solution is an iterative process requiring careful prompt engineering. The connection to human-human collaboration is clear: until the software engineer does not understand what the researcher exactly wants, she/he will probably deliver unsatisfactory solution

The response is shown in Appendix A and the corresponding plot is depicted at right-hand side of Figure 2. When a project is migrated from one programming language to another, this ability of ChatGPT could limit the related efforts/expenses dramatically.

In this case, the transpilation to R was not successful as we again got a problem with the dimensions matching of the input of ClaytonCopulaMLE and output of ClaytonSample. We were also unsuccessful with copy-pasting the error message to ChatGPT. Clearly, we could be more precise by specifying these dimensions. However, we were deliberately underspecifying the task in order to illustrate what separates successful solutions from unsuccessful ones.

To resolve the task, we finally used a rather “brute-force” approach consisting of copy-pasting the whole code in Python to ChatGPT and asking it for the same code on R. The response is shown in Appendix B and the corresponding plot is depicted at the bottom of Figure 2. Another way we obtained a working solution was to feed ChatGPT with explicit information about the dimensions of the inputs and outputs, i.e., using a similar approach as before by adding “U = ”. On the one hand, this approach is more elegant then the brute-force one, on the other hand, the brute-force approach allowed us to get a working solution in less time. A choice between them thus depends on the user’s priorities.

Basically we now have an implementation of our example task in every language supported by ChatGPT. Note that an implementation of sampling from the Clayton copula in Python appeared relatively recently as a serious effort presented in Boulin (2022). Here, we got an equivalent implementation completely created by an AI. Clearly, with the abilities of ChatGPT, achieving a solution in distinct programming languages will become much less valuable.

3.6 The parallelization

Note that as the codes for the following tasks in Python and R were rather long, we present only the MATLAB versions.

3.6.1 CPUs

We directly got a working solution, which generated the plot shown at the lefthand side of Figure 3. Let us highlight several points:

• Being in the context of the conversation, we could afford to be extremely concise and use just one word (“that” in the first sentence) to point at the functionality we request; natural and time-saving.

• We just prompted ChatGPT with a very high-level request, Create a demonstration, and got exactly what we wanted. Originally, we used, say, the traditional way of thinking, which involves being very specific when designing a task. For example, “create an array for time measurements,

measure time for this and that, store it in the array and then plot it in the way that...”. Since such tasks probably occur many times in ChatGPT’s training data, overly specifying is not necessary, which, again, can save a lot of time.

• Notice the 4 workers in numWorkers = 1:4. It is a common number of CPUs of an average office PC. This number is thus not just a pure random guess, but rather a most occurring one in ChatGPT’s training data.

Note that apart from our office PC, we executed the code on an available server with 32 CPUs, which generated the plot at the right-hand side of Figure 3. As only 20 theta values (thetas) are considered, we observe that the improvement in time does not continue from the 21st worker onwards.

3.6.2 GPUs

We got two pieces of code as response. First, ChatGPT provided an optimization of ClaytonSample for GPUs.

Then, a code for its demonstration followed.

After executing the latter, the output was:

CPU time: 0.17493 seconds

GPU time: 0.093107 seconds

As can be observed, the optimization of ClaytonSample, ClaytonSampleGPUs, is based on the addition of gpuArray, which assures that the sampling from the standard exponential and gamma distributions as well as the remaining computations are performed directly on available GPUs. The outputs U and V are then gathered from the GPUs onto the client by gather. As the output of the demonstration part of the code shows, this roughly halved the non-optimized run-time.

Apart from a more efficient implementation, ChatGPT also saves our time by exempting us from inspecting whether or not each involved function is supported on GPUs. We should also not forget the educational aspect of the matter. As these optimization techniques are rather advanced, an inexperienced user genuinely learns from these outputs (having in mind they might not always be correct). For example, without explicitly mentioning it in our prompt, the values of theta and n are stored in separate variables before they are used in ClaytonSample and ClaytonSampleGPUs. This belongs to proper coding techniques.