barefootstache @barefootstache@social.cleverlibre.org

#DailyBloggingChallenge (155/200)

To actively scrape a #website one employs either an extension or uses the console.

Here the difference is where and who maintains the code. The benefit of using the #console is that one is browser agnostic and still can keep a level anonymity. Whereas with an extension could be used as a fingerprint marker.

E.g. if using the #Tor browser one should not diverge from the installed extensions, since one will easier identified compared to the herd. Using the console would be preferred in this case.

On the flip side using an extension voids the need to copy and paste the code into the console every time.

#DailyBloggingChallenge (154/200)

To passively scrape a webpage one uses automation tools, ideally headless browsers like #Selenium or #Puppeteer. Of course one can use any tool that is typically used for #e2e testing in the #browser.

The biggest obstacle for passively scraping is dealing with either #captcha or #cloudflare.

There are options to use captcha farms for a small monetary fee. And Cloudflare can be over come by IP hopping.

In general, passively scraping only works on websites that were poorly configured.

#DailyBloggingChallenge (153/200)

There are two main ways to #scrape a #website, either actively or passively.

_Active scraping_ is the process of using a trigger to actively scrape the already loaded webpage.

_Passive scraping_ is the process of having the tool navigate to the webpage and scrape it.

The main difference is how one is getting to the loaded #webpage.

#WebsiteScraping

#DailyBloggingChallenge (152/200)

Not only hardware is a concern, though also internet speed. Lots of websites use some kind of media like images or videos and many don't convert these to slow internet friendly speeds.

For images WEBP suffices and for videos a bit rate of 8Mbits.

#DailyBloggingChallenge (151/200)

Lots of websites these days are being first built on the client. This can easily be checked when downloading the #website does not align with the #HTML from the inspector.

This has the benefit for the provider to save transfer cost, though on the flip side, the client will need to have a specific amount of #hardware to successfully render the site.

#DailyBloggingChallenge (150/200)

Designing themes with #Vuetify is fairly straightforward, the difficulty is creating a #design or color palette in the first place.

In this approach the "import full palette" method was chosen. This consists of importing the color palette and assigning each color an unique identifier. The type `ThemeDefinition` exists to help with naming conventions. The addition name to add is `accent` which should fit well with the `primary` and `secondary` colors.

Later when the #theme is being built one can directly choose from the palette.

#DailyBloggingChallenge (149/200)

#TIL that the female #human has on average proportionally shorter legs in respect to their body height compared to their male counterparts.

This could also be the explanation why on average the female human is biologically predisposed to be able to touch their feet with their hands while keeping their legs straight.

#biology

#DailyBloggingChallenge (148/200)

Currently just imagining the idea of surpassing #weeklyOSM $w$ blog count with the DailyBloggingChallenge $d$.

To calculate the weeks number $x$, one sets $w(x) = d(x)$ with $w(x) = 700 + x$ and $d(x) = 148 + 7x$. This makes $x = 92$ weeks or 645 days until surpassed.

So just before #weeklyOSM800 (relatively speaking) edition, the DailyBloggingChallenge would overhaul it.

Well before fantasizing of hypothetical goals, I should stick with the current goal of 200.

#maths

#DailyBloggingChallenge (147/200)

As an active participant of the @weeklyOSM project which is celebrating its 700th weekly news update, one gets to admire a long lasting community project.

This would put the first edition almost 14a ago, which is only a couple years after the #OpenStreetMap project started.

There are a lot of people working behind the scenes of gathering the news stories, writing up a small summary, translating these into the variously languages, proof reading, and finally publishing at the end of the week.

#weeklyOSM700

#DailyBloggingChallenge (146/200)

One downside of #Croatia opening its borders to be part of #Schengen is that when flying into #Zagreb with #RyanAir is that there is a high chance that it will continue its flight outside of Schengen. This means that the plane will park at the international terminal making it more convenient for the upcoming passengers. Though the current passengers get to be conveyed with buses to the terminal.

On the flip side when flying out to Schengen area is similar as before with access to most amenities.

They put border patrol right in front of the gate. Instead of having it right after security which most airports follow.

#FunOfFlying

#DailyBloggingChallenge (145/200)

The `TagTable` has a `photo_ids_list` which contain each photo or video with the specific tag. Further each `id` is either prepended by `thumb` for photo and `video-` for video. The `id` itself is saved as a hexadecimal number.

Once one has the `id`'s as a decimal number, one can use it to search through either the `PhotoTable` or `VideoTable`.

#DailyBloggingChallenge (144/200)

One downside of using #Shotwell as a photo and video manager is that it isn't that straight forward to extract tags from videos as it is with photos. With photos one can grab the data directly with an #exif tool.

After going through the source code and battling with #sqlite and #bash, I got a script that extracts tags from both media formats.

The nice thing is that it is more performant than the previous script I was using only for images.

@mdione in term of tiles.

And the 1m or 5m was done in north Europe. Don't know how valid the 1m is in OsmAnd since one needs to zoom into the max and then hold down on the "zoom in" or "zoom out" button and choose 400%.

#DailyBloggingChallenge (143/200)

TIL how zooming on maps work. The fascinating part is how much data needs to be store to make it feel seamlessly.

In (1) they work with an initial presetation of Earth on a 256px x 256px square for a zoom of 0.

When zooming in by one level the spuare length is doubled and thereby the Earth is being presented on 4 times 256px x 256px square for a zoom of 1.

This can be generalized to $4^{zoom}$ for square count and $squareLength_0 * 2^{zoom}$ for square length.

#LeafletJs states in (1) that tile servers max out at a zoom level of 18, whereas #OpenStreetMap's website goes to zoom 19. At zoom 19 the legend distance is at 5m. On #OsmAnd the max zoom achieves a legend distance of 1m.

One can just imagine how much data is being stored/provided to achieve such a zoom level of detail.

References:

(1) https://leafletjs.com/examples/zoom-levels/

#DailyBloggingChallenge (142/200)

TIL that the #Jenga instructions differ depending on the language.

In English it says that after taking a block from the tower to place it on the top. Whereas in German it adds the additional information that it needs to be perpendicular to the blocks on the layer below it.

That additional info reduces the amount of choices from infinity down to a minimum of three as long as the distance from the center is not considered.

Let the smallest area of sides be denoted front $F$ and back $B$ and the center line as $C$. Then typically the distance of $|F - C| = |B - C| = b/2$ with $b$ as block length. Though, no where in the rules does it say that this equation has to hold up. Let $R := \{<,=,>\}$ be a relation, then $|F - C| R |B - C|$ is still true with $|F - C| \in [0, b]$. Since the distance $|F - C|$ from the center is in a non one element interval, there are infinite solutions. (1)

To get the minimum of three, one needs to count the amount of ways one can place a block, which is three: smallest area side, mid area side, and largest area side.

Additionally to this placement there is a rotation variable $r$ which has to be 90° or $\pi/2$. Otherwise without this restriction $r \in [0, \pi)$ and this would again give infinite solutions.

So going from the English instructions which would create infinite placement options down to German instructions, without (1), would give 3 placement possibilities.

Technically speaking, placing the blocks on the largest area side is a house rule.

#maths