Advanced monitor data (xarray) manipulation and visualization¶

In the previous tutorial, we have demonstrated the basics of data manipulation and visualization. As discussed, in Tidy3D, we represent multi-dimensional data (such as arrays) using xarray.DataArray objects. It's important to note that the xarray package provides a lot of functionalities that can greatly simplify our data postprocessing and visualization when used appropriately.

In this tutorial, we will first briefly review some basic xarray features and then delve into some advanced features such as performing interpolation and integration. This tutorial is designed for users who are already familiar with Tidy3D's basic workflow and would like to learn more about data manipulation and visualization. If you are very new to Tidy3D, we suggest checking out the quick start, the first walkthrough, and the working principles tutorials first.

It's important to note that the xarray library contains many more functionalities that are not demonstrated here. We encourage users to reference the documentation to see what is possible. We also recommend users to watch the introduction video on xarray if you would like to learn more about it. After overcoming the initial learning curve, our users all find data manipulation with xarray to be extremely flexible and efficient.

Load Simulation Data¶

Since this tutorial is focused on data analysis, we will skip the simulation setup part and only load the simulation result from a previously obtained hdf5 file. We load the hdf5 file to a variable sim_data using the from_hdf5 method of SimulationData. This is completely analogous to the situation where you just finished running a simulation.

The loaded data contains two parts: simulation and monitor_data. simulation is simply the original Simulation. We can use it to visualize the simulation in 3D for example.

As we can see, this simulation contains a 1 by 2 MMI power splitter.

monitor_data, as the name suggests, contains data from all monitors defined in the simulation. monitor_data is a dictionary with the keys being the names of the monitors. Let's print out the keys to see what they are.

Here we have three monitors named Flux monitor, Mode monitor, and Field monitor. In the following sections, we will examine them individually and use them to demonstrate various functionalities of xarray.

FluxMonitor Data¶

We can access the data for each monitor by indexing into the SimulationData with the monitor .name. The monitor dataset contains one or more xarray.DataArray objects storing the actual raw data. For the flux monitor data, we can access the raw flux data as a function of frequency with .flux. As most data are multidimensional, it's often very helpful to print out the data and directly inspect its structure.

Here we can see that the flux is an array with only one coordinate f, which is frequency in Hz. The unit of flux is W.

The data array has .values and .coords. The .values are just the raw, unlabelled data. The .coords are a dictionary mapping each dimension of the data to a set of data. For example, mapping the flux values to a specific frequency. We can inspect them by printing them out.

xarray allows us to quickly plot the data. In this case, the plot will just be a 1D line since the data only contains one coordinate. Note that the x and y axis labels are automatically created accordingly.

In many cases, we would like to customize the plot. For example, the transmission is often plotted in dB scale. Instead of frequency in Hz as the x axis, wavelength in micron is more standard. In addition, we might want to plot the curve in a certain style. This can be achieved by first extracting the data from the DataArray and manually plotting them using matplotlib or other visualization library of choice.

Saving Data as a CSV File¶

After running a simulation, often we want to save the data for future use to avoid rerunning the same simulation. In Tidy3D, after a simulation is finished, the hdf5 data file will be automatically downloaded to the specified path. We can load it in the future as demonstrated at the beginning of this notebook.

Sometimes users might prefer to only save certain data in other formats such as csv files. Here we demonstrate how to save the transmission flux spectrum. First we define a numpy array to store the data and then use the savetxt method to save it as a csv file. This csv file will contain two columns with the first column being the frequencies and the second column being the flux.

ModeMonitor Data¶

The mode monitor data contains the complex effective indices and the complex mode amplitudes at the monitor position calculated by mode decomposition. First, let's look at the data structure of the complex effective indices n_complex. This data contains two coordinates: f and mode_index, both of which are specified when defining the ModeMonitor in the simulation. In this case specifically, we solved for 3 modes at 50 frequency points for this monitor.

Data Selection/Slicing¶

For multidimensional data, it's important to slice them correctly to extract the data that we are looking for. For example, assume that we want to plot the effective indices as a function of frequency for each mode. This can be easily achieved by using the sel method of xarray.

After slicing the data, we can directly plot it on a matplotlib axis so we can style the plot after plotting it.

In sel, we can specify multiple coordinates at the same time. For example, we want to extract the effective index of the first mode (index 0) at the highest frequency.

Equivalently, the same can be achieved by using isel, which stands for index selection. Unlike sel, which selects at specific coordinate values, isel selects at specific coordinate index. For example, the highest frequency is the index 0 of the frequency coordinate. Therefore, isel(mode_index=0, f=0) will give the same result as sel(mode_index=0, f=freqs[0]).

Besides the effective index, ModeMonitor is primarily used to calculate the transmission of certain modes in certain directions. We can extract the complex amplitude and square it to compute the mode transmission power.

In addition to the support of standard mathematical operations (+, -, *, **), xarray is also compatible with numpy so we can directly apply numpy operations such as square to an xarray DataArray. As a demonstration, we extract the transmission of the fundamental mode in the positive direction (sel(mode_index=0, direction='+')), take the absolute value (.abs), and square it using numpy. Then we can plot it in dB scale. The plot is similar to the previous plot from Flux monitor, which is expected as the waveguide is single mode and most of the transmitted power is at the fundamental mode.

Note that in Tidy3D, we implemented additional convenient functionalities that are not avaiable in the origianl xarray. Taking the absolute value (.abs) is one example. One can achieve the same result by using abs() on an DataArray.

FieldMonitor Data¶

Finally let's examine the data in Field monitor. This dataset contains all electric and magnetic field components: Ex, Ey, Ez, Hx, Hy, and Hz. Let's use Hz as an example to explore some visualization possibilities.

This array has 4 coordinates: three spatial coordinates and one frequency coordinate.

We can slice the data in the same way using sel and(or) isel. Let's look at the Hz field at the first frequency (1500 nm) and z=0 plane. When using sel, it's important to note that the specific coordinate value might not be available in the data. For example, the z coordinate contains four values, namely -0.11, -0.022, 0.066, and 0.11. That is, z=0 is not available. We can specify method='nearest' in the selection such that it automatically extracts the data at the nearest plane, e.g. z=-0.022.

After slicing, the data is 2D so it can be plotted as a false color plot. When using the plot method, we can specify which coordinate is plotted on the horizontal axis, which coordinate is plotted on the vertical axis, what colormap is used, min and max values of the color scale, etc.

Similarly, we can easily extract and plot the field at 1550 nm.

Data Interpolation¶

The field data are most likely located in a nonlinear grid. However, xarray allows us to interpolate data at arbitrary coordinates easily. As a demonstration, we define a linear grid and interpolate the Hz field to this grid by using the interp method.

Similarly, we can save the field data as a csv file for future use.

Integration¶

Sometimes the postprocessing of the simulation data requires the calculation of certain integrals. A common example is the calculation of the overlap integral of two modes. For instance, when designing an edge coupler, we might want to calculate the overlap integral of the fiber mode and the waveguide mode. In Tidy3D, we provide a convenient function outer_dot that does this integral.

To demonstrate this, we load data from two hdf5 files. They contain the field profiles of a Gaussian mode and a waveguide mode.

Before calculating the overlap, let's plot the fields to visualize them.

From the field profiles, we can see that the waveguide mode and the Gaussian mode have a similar mode area so they should have a significant overlap.

Now we calculate the overlap using outer_dot and plot the result in dB.

For demonstration purposes, we can also perform the integral manually using the integrate method in xarray. The overlap integral is given by

$$ \frac{1}{4} \left(\mathbf{E}_1 \times \mathbf{H}_2^* - \mathbf{H}_1 \times \mathbf{E}_2^* \right) dA, $$

where $\mathbf{E}_1$ and $\mathbf{H}_1$ are the electric and magnetic fields of the first mode, $\mathbf{E}_2$ and $\mathbf{H}_2$ are the fields of the second mode, and * denotes complex conjugation.

One complication here is that the fields of the two modes might not share the same grid as they can be calculated from different simulations. The computation between different arrays does require them to share the same coordinates. In this situation, we can use the interp_like method to easily interpolate one array to the grid of the other array.

As expected, the result calculated this way is the same as that calculated from outer_dot above.

Besides the overlap integral shown above, in other example notebooks we also demonstrated how to perform integrals to calculate the cavity mode volume and the nonlinear feedback parameter.

Closing Remarks¶

In this tutorial, we demonstrated various futures commonly used in Tidy3D's data postprocessing and visualization. We used data from FieldMonitor, FluxMonitor, ModeMonitor, and ModeSolver as examples. However, the methods are general and equally applicable to data from DiffractionMonitor, FieldProjectionCartesianMonitor, and so on.