Lx=d: SundayResearch

Overview This is a program for migrating from a Wordpress with qTranslate to Wordpress multi-site. It removes the language-specific contents and tags attached by qTranslate plugin. Keywords: qTranslate, qTranslate-X, multi-site, multi-language, Wordpress Introduction Ten years ago from now (abound 2013), it is common that the cheapest web hosting service had 0 SQL databases, and even the next level service with database had up to 2 SQL databases. Therefore, if you want to use Wordpress in multiple languages, qTranslate multilingual plug-ins https://qtranslatexteam.wordpress.com/ was a common solution to reduce the number of database use. How does the qTranslate plugin work? The qTranslate plugin puts special tags ([:ja], [:], etc.) in the text. The plugin will retrieve the specific language text based on the tag. Advantages and disadvantage of the qTranslate plugin The biggest advantage of the qTranslate plugin is that it needs only one database even if it is multilingual. Ten years a...

I need to have a sha1 digest from both C++ code and python code. Here is a code snip to match both results. This code avoids a potential problem that the digest has some 0s on top of the digest array element. This doesn't matter if you stick to one implementation, but just in case, you need to match two worlds: C++ and python, this code might be useful. /// get sha1 digest as a std::string /// /// \param[in] mes message to be hashed /// \return digest string std::string get_sha1_digest(const std::string& mes) { boost::uuids::detail::sha1 sha1; sha1.process_bytes(mes.c_str(), mes.size()); const int DIGEST_SIZE = 5; unsigned int sha1_hash[DIGEST_SIZE]; sha1.get_digest(sha1_hash); std::stringstream sstr; for (std::size_t i=0; i < DIGEST_SIZE; ++i) { sstr << std::setfill('0') << std::setw(8) << std::hex << sha1_hash[i]; } return sstr.str(); } This function's output matches with the fo...

I use Jenkins  for a nightly automated test. In the test I use valgrind  for memory leak detection and has some graph of number of errors. So I would like to use plot plugin . From the example plot image , it is clear we can have a graph with multiple lines. However, how to do it was a bit ambiguous to me from the web page, the example perl script, and the help (in the plot plugin). The key idea is I need to have multiple data files for multiple lines. For example, in the build I made the following property data files. Each file represents one data line. valgrind_trunk_result.definitely.property valgrind_trunk_result.indirectly.property valgrind_trunk_result.possibly.property The contents of each file is only one line. For example,  valgrind_trunk_result.definitely.property is: YVALUE=0 This file is put under the directory of ${WORKSPACE} where the environment variable "WORKSPACE" is given by jenkins. Figure 1 shows my plot plugin setup in th...

Abstract I voluntarily work on for free mathematics material translation for everyone. I have three main tasks in my workflow of this work: 1. script translation on a srt file, 2. dubbing the video, 3. subtitle generation. I found the subtitle timing generation is a time consuming task, so I want to reduce this. When I generate a subtitle, I already have the translated script and its video sound. So, I try to use these data to semi-automate the subtitle timing generation. This time I use the YouTube's transcript function to generate the subtitle timing. This can reduce the time of timing generation task. I implemented a srt file to text file conversion script since YouTube's transcript function requires text format data. YouTube's transcript function performs  not only the timing generation, it also edit the lines (put some newlines). Therefore, I implemented subtitle line concatenation script, too. One experiment shows that whole manual work took 4.5 hours to generate th...

My last blog entry, I briefly talk about what is Scratch. There are ``events'' and that is the trigger of the program. In this article, I would like to talk about one of my students who made a character animation program by Scratch. Scratch provides key events. When I push the right arrow key, then ``right arrow key push event'' is triggered. When that event happened, I add 10 to the x coordinate of the cat. This means, when the right arrow button pushed, the cat move to the right for 10 steps. If I add more programs like the up arrow moves the cat to upper direction, I can control the cat position by the arrow keys. This is a cat control program. Today, my student used a dragon. He wrote the same program with the cat. Then, the cat and the dragon move exactly the same way. Of course they should. A current computer is very fast, very precise, and very stupid. It does only what the developer wrote. My student asked me, how he can make the dragon faster. I answered, i...

I teach Scratch [1] , a computer programming language, to 10 to 12 years old students from last year. Sometimes they show me an interesting creative idea. I would like to write one of them here. Before I explain what my students did, I will explain what is Scratch. First I will explain it in a conceptual way, then with an example. If someone can understand a conceptual explanation, he/she can apply the idea to many cases, but this is a bit difficult since their understanding must be deep. An explanation by example is rather easy to understand since it is shallow, that means you don't know it is still true in other cases. So there is a trade-off. Scratch is a computer language, but also a programming environment.  In the environment, the developer writes event driven programs to control many sprite [2] characters. You can think each sprite character an object. Each event invokes a program and each program runs in parallel. Maybe this explanation doesn't make sense for who...

I always try to simplify a command line parser implementation regardless any programming language: C++, bash, ... To do that, I restrict the command line option to a regular form only. All my arguments should be '-arg_key value' form. Even I want to specify a file name, my program needs an argument key, e.g., '-in_file input_filename. If you do in this way, each command line argument has the key, so I can put all the command line options to a map. This makes the command line parser simple. I usually don't need getopt library. Also I try to simplify the command line option support, means smaller number of command line options, and use a config file, which contains 'key = value' lines. This has two advantages: you can support negative values without confusing the command line option, easy to reproduce the test case. However, this method still has a problem when the command line option includes white space. If I could, I will only try to use config files,...

Last article, I showed a simple enumeration generator. But I needed one more flexibility. In my job, I use GPU, that is a fast processing unit, but the available memory size is one order of magnitude small compare to a decent workstation. (E.g, GPU's memory size is 2GB to 6GB, a decent workstation can have 24GB to 256GB main memory.) In this pseudo code example, the unit of memory size is GB. 64GB or 512 GB are too much to the current stare of the art GPUs in 2013.  512GB is too much for the best workstation. Therefore, our product uses a cluster, many workstations are cooperates to do a single job. Almost every customer wants to see how our products scales regarding to the number of workstations. Because they have their needs and they want to know how many workstations and how many GPU are needed for their task. Therefore, we need to demonstrate how the performance changes depends on the number of nodes. Here one more parameter, the number of nodes are added as the following. ...

PIL and numpy When I ran this program on my data files, I found the processing time is around 6 seconds, the memory consumption size is 230MB on a 1024x1024 size image. When I processed images resolution of 3840x2160, it took 263 seconds and 2.3 GB memory is consumed. The difference of these resolutions makes only eight times different number of pixels. But the processing time is increased more than 40 times. In my program I only use three buffers for processing, my first estimated minimal program sizes are 10MB for 1024x1024 resolution and 72MB for 3840x2160 resolution. However, the `top' reported 30 times more memory size. When I profiled the program, the most of the time is consumed by the tuple construction (RBG value) and abs function. Therefore, I tried to use numpy to vectorize these code. A table below shows the result. My test environment of Intel Core i7-2720 2.20GHz Linux (Kubuntu 12.10, kernel 3.5.0-27), Python 2.7. +-----------+----------------------------------...

Abstract: Writing image comparison tool with Python PIL. Python PIL module Python Imaging Library (PIL) is a useful Python module to process image files. This time I have a situation that I have different image file format files But the contents must be the same. For example, I wrote a image generation tool and I want to test it. I compress the reference images, but my program produces images with non-compressed image file format. I can use convert (ImageMagick) tools, though this time, I just would like to try a new tool. You can find my image comparison tool here.

Abstract: Using mathematical objects often makes a program simpler. This time I have such experience and write it down here. Mathematical object and programming In a program test, we often need to generate a combination of input parameter sets. One of the most easy method to generate a combination is using nested loops. In this article, I use pseudo code based on the Python language. I will provide the real implementation of the program in the appendix. For example, we have following two parameter sets:  data_size_list = [ 5, 64, 512, ]  screen_resolution_list = [     '2560x1440', '3840x2160', ]. The following program can generate the combination of them:   for d in data_size_list:     for s in screen_resolution_list:       print_comb(d, s) # output This method is simple and straightforward, however, less flexible in some cases. For example, we don't know which sets are necessary to generate a combination whe...