Lx=d: SundayResearch

Overview This is a program for migrating from a Wordpress with qTranslate to Wordpress multi-site. It removes the language-specific contents and tags attached by qTranslate plugin. Keywords: qTranslate, qTranslate-X, multi-site, multi-language, Wordpress Introduction Ten years ago from now (abound 2013), it is common that the cheapest web hosting service had 0 SQL databases, and even the next level service with database had up to 2 SQL databases. Therefore, if you want to use Wordpress in multiple languages, qTranslate multilingual plug-ins https://qtranslatexteam.wordpress.com/ was a common solution to reduce the number of database use. How does the qTranslate plugin work? The qTranslate plugin puts special tags ([:ja], [:], etc.) in the text. The plugin will retrieve the specific language text based on the tag. Advantages and disadvantage of the qTranslate plugin The biggest advantage of the qTranslate plugin is that it needs only one database even if it is multilingual. Ten years a...

I need to have a sha1 digest from both C++ code and python code. Here is a code snip to match both results. This code avoids a potential problem that the digest has some 0s on top of the digest array element. This doesn't matter if you stick to one implementation, but just in case, you need to match two worlds: C++ and python, this code might be useful. /// get sha1 digest as a std::string /// /// \param[in] mes message to be hashed /// \return digest string std::string get_sha1_digest(const std::string& mes) { boost::uuids::detail::sha1 sha1; sha1.process_bytes(mes.c_str(), mes.size()); const int DIGEST_SIZE = 5; unsigned int sha1_hash[DIGEST_SIZE]; sha1.get_digest(sha1_hash); std::stringstream sstr; for (std::size_t i=0; i < DIGEST_SIZE; ++i) { sstr << std::setfill('0') << std::setw(8) << std::hex << sha1_hash[i]; } return sstr.str(); } This function's output matches with the fo...

Abstract I voluntarily work on for free mathematics material translation for everyone. I have three main tasks in my workflow of this work: 1. script translation on a srt file, 2. dubbing the video, 3. subtitle generation. I found the subtitle timing generation is a time consuming task, so I want to reduce this. When I generate a subtitle, I already have the translated script and its video sound. So, I try to use these data to semi-automate the subtitle timing generation. This time I use the YouTube's transcript function to generate the subtitle timing. This can reduce the time of timing generation task. I implemented a srt file to text file conversion script since YouTube's transcript function requires text format data. YouTube's transcript function performs  not only the timing generation, it also edit the lines (put some newlines). Therefore, I implemented subtitle line concatenation script, too. One experiment shows that whole manual work took 4.5 hours to generate th...

PIL and numpy When I ran this program on my data files, I found the processing time is around 6 seconds, the memory consumption size is 230MB on a 1024x1024 size image. When I processed images resolution of 3840x2160, it took 263 seconds and 2.3 GB memory is consumed. The difference of these resolutions makes only eight times different number of pixels. But the processing time is increased more than 40 times. In my program I only use three buffers for processing, my first estimated minimal program sizes are 10MB for 1024x1024 resolution and 72MB for 3840x2160 resolution. However, the `top' reported 30 times more memory size. When I profiled the program, the most of the time is consumed by the tuple construction (RBG value) and abs function. Therefore, I tried to use numpy to vectorize these code. A table below shows the result. My test environment of Intel Core i7-2720 2.20GHz Linux (Kubuntu 12.10, kernel 3.5.0-27), Python 2.7. +-----------+----------------------------------...

Abstract: Writing image comparison tool with Python PIL. Python PIL module Python Imaging Library (PIL) is a useful Python module to process image files. This time I have a situation that I have different image file format files But the contents must be the same. For example, I wrote a image generation tool and I want to test it. I compress the reference images, but my program produces images with non-compressed image file format. I can use convert (ImageMagick) tools, though this time, I just would like to try a new tool. You can find my image comparison tool here.

My friend, Christian told me a story about what is int(-1/2), 0 or -1? Christian found a problem when he explained a binary search program. When he computed the lower bound, but, the entry is not in an array.   mid = (left+right)/2 When left = -1, right = 0. The mid is 0 in C, C++, and Java. The mid is -1 in python and ruby. I was also surprised this simple looking code depends on languages. This is based on rounding method. C, C++, Java, and elisp does round to zero (truncate), therefore, this is 0. python, ruby does round down (floor), therefore, this is -1. When we think about modulo, python and ruby holds    x = x/y + x%y condition, but not C++, Java. The modulo operation also depends on the languages when minus value is considered. One more note, when I looked up binary search on the web, this mid code has overflow problem, so, it should be   mid = low + (high - low)/2; References http://en.wikipedia.o...

How to access to self.face_idx_list . extracting a numpy.int64 type failed. The difference between this article and the last one is numpy.array contains numpy.int64 instead of numpy.float64. We can access numpy.float64 value as the following: vec[i] = boost::python::extract< float >(     float32_3_obj.attr("__getitem__")(i)); I.e., we can convert numpy.float64 value to float value by boost::python::extract< float >(f) . However, when we try to convert numpy.int64 to int as the following code, it fails. vec[i] = boost::python::extract< int >(     int32_3_obj.attr("__getitem__")(i)); We can not convert numpy.int64 by boost::python::extract< int >(i) . We will get the following error: TypeError: No registered converter was able to produce a C++ rvalue of type int from this Python object of type numpy.int64． That's strange. float is OK, but int isn't.  I looked up the web, there are some articles regarding with thi...

How to access to the self.vertex_list. This time I will explain how to access to the self.vertex_list of TriMesh. This is a list of length three numpy.array. We have already seen how to get the python list (dict's keylist) in my former article http://shitohichiumaya.blogspot.com/2010/08/boostpython-how-to-pass-python-object.html . How to access to the float numpy.array element is the following. void print_float32_3(boost::python::object const & float32_3_obj) {     // check sequence length is 3 or not.     if(boost::python::len(float32_3_obj) != 3){         std::string const objstr =             boost::python::extract< std::string >(                 boost::python::str(float32_3_obj));         std::cerr << "print_float32_3: arg is not a float[3] obj ["                   << objst...

The python code of the last article contains a simple triangle mesh (TriMesh) and sets some data to the object. The mesh has only two triangles that forms a rectangle. It's not so exciting scene, but it is sufficient as an example. The scene is shown in the following image. A camera looks up the rectangle. Figure 1. TriMesh scene example (one rectangle) This article explains how to access to the three members of TriMesh. Each member of a python object is an attribute, there is a way to access to any attribute in general, but I think it is complicated. I assume the programmer has an knowledge about a specific python class. For example, I know TriMesh object has three member variables (three attributes). self.__material_name : a string, but this is a private member. We should access to this member via get_material_name() method. self.vertex_list:  A list of numpy.array, that contains length three numpy.float64. self.face_idx_list : A list of numpy.array, that conta...

Once I explained how to pass a python build-in object (dict, list, and string) to C++ code using python.boost. http://shitohichiumaya.blogspot.com/2010/08/boostpython-how-to-pass-python-object.html This time, I will explain about how to pass a user defined python object to C++ via python.boost. I will also explain why numpy.array's numpy.int64 object casts an error when I do extract< int >. This casts a TypeError (TypeError: No registered converter was able to produce a C++ rvalue of type int from this Python object of type numpy.int64()). Then I will show you how to solve this problem. Here is an python example code. ----- # # test pass object 2 module, python side implementation # Copyright (C) 2012 Hitoshi # import numpy import passobj2_mod class TriMesh(object):     """TriMesh: simplified triangle mesh primitive for python.boost     demo"""     def __init__(self, _mat_name):         """default construc...

This article is again about programming language, so if you are not interested in this area, see you next time. My first python project is a test suite. I like Martin Fowler's articles, so, naturally, my first program is a test. My project is usually started with test, documentation, and interface design, yet incremental. This is about python's os.environ and LD_LIBRARY_PATH. This is a follow up of the boost.python's story. My development environment is Linux, but later I have a plan to go to Windows also. In python interpreter, we can change the environment variables through os.environ. For example, if you want to change the LD_LIBRARY_PATH:   os.environ['LD_LIBRARY_PATH'] = '/some/directory/lib' I thought this is all for LD_LIBRARY_PATH, but I hit a wall. My department policy doesn't allow to have administrator right and Internet access to the developers. Therefore, if I want to use something not in my computer, for example, boost or python, ...

This is a programming language story. If you are not interested in such theme, see you other time... I usually use ruby for scripting, however, in industry python is quite widely used. When I was a student, I was just interested in programming languages, yet I just interested in them for a few weeks and I did not use most of them. But, recently I even did not look into, I feel I become old... This time, I will try what so called ''python.'' (In my Japanese Web, this is ''皆のすなる python というものを,'' this is beginning of Tosa-nikki by Kino Turayuki, established around 935. Sadly, I can not translate this well by my poor English skill.) First I read a book, Learning Python by Marc Lutz. I took around two weeks, it is fun and I find python interesting. Then this week, I started to implement a program. There are a lot of introduction web pages of python, so, it is not worth to add something similar to the net, the readers will be also bored such blog entry. ...