kemuri 🦀 @kemuri@social.cleverlibre.org

下午法律培训，律师说起公安机关用的最多词就是“粗糙”，指公安作风粗暴，疑罪从有，只记录对自己有利的证据，多以威胁恐吓为手段。

律师说起应对方式大概就是，少说少错，一问三不知，不认识不知道没参与，不要乱承认，也不要相信公安说的任何“交易”(坦白了就能早出去、少判点等等，他们没有这个权限）或诱导(说别人都坦白了都说是你，让你咬他们)。中心思想就是坦白从宽牢底坐穿，抗拒从严回家过年。

还特别提到，群聊内容现在经常作为调查取证重点，今年某起聚众抗议事件的案子，诉领头者的80%的证据都是群聊记录。
但又说到，聊天记录等电子证据取证作证的方式，竟然是交可以登录微信的原手机上去手动查，而不是直接从运营商或微信后台调取(仅限民事案件)。所以建议经常清理手机和聊天记录，不该留的别留。

カリカリのおかわりを待っている猫

#キュウちゃん #kyuchan #cat #mastocats

Christmas Hedgehog #hedgehog

#论文导读 @mature

剩饭炒炒还能吃——chatGPT

想必各行各业的大家对 chatGPT 都有所耳闻了吧。有拿它编论文的有拿它写剧本的有拿它改代码的，还有打算拿它代替搜索引擎的，总之被玩出了花。谁看了不惊呼机器早晚反抗人类暴政呢？

聪明的你是否会好奇这背后藏着么大秘密？为什么她会有如此惊人的效果。

顾名思义，chatGPT由两个部分组成，chat和GPT。看到这里，读者想必有了一些猜测。没错，chat是它的训练手段，GPT是它的本质。

那么什么是GPT呢？GPT是一个语言模型，它的初代诞生时间稍早于BERT，然后在之后的很长一段时间内被BERT掩盖住了光芒。原因大概是不管GPT几，对于一般人来说train起来实在是太困(烧)难(钱)了。GPT和BERT一样，都是基于Transformer的，不同的地方在于前者是用decoder train的，后者是用encoder train的。因为encoder和decoder看到的序列的差异导致了decoder train起来更难一些，但是一旦train好的话上限其实是更高的。

再简单说一下语言模型的运作方式，就是你先喂一小段文字进去，模型会根据你喂进去的文字去预测下一个最有可能出现的词(token)，再把这个词跟之前的文字一起喂进去，吐出下一个最有可能出现的词。就这样一直循环。所以理论上你让它吐得越多它就会越离谱……
所以你以开始喂进去的那一小段话我们就可以把它叫做prompt，提示语。模型在做的事就是不断地续写。根据prompt的类型的不同，续写的内容可以是问题的答案，也可以是一个命题作文。

然后就到了chat的部分。openAI应该是专门写了一个chatbot用来给数据标注人员做交互。可能这样比较有参与感，写出来的语料质量也更高吧。整个训练过程叫做Reinforcement Learning from Human Feedback (RLHF)，由3个步骤组成：
-第一步是准备一个prompts的数据集，标注人员给prompts写desired outputs，模型也会产生一些辅助的outputs给人修修改改。用这样的prompts-outputs数据对就可以先finetune一下GPT了。
-第二步：经过第一步之后模型已经能吐出一部分回答了，但是不是所有的回答都特别理想，这时候标注人员可以对拿到模型对同一个prompt生成的不同回答打分。根据prompt-output-score这样的数据，又可以train一个reward model（RM）来代替人工打分的这个过程。
-有了自动生成答案和自动打分的模型之后我们就可以让语言模型自己去优化了，这就是第三步。这是一个强化学习的过程，目标是让RM打的分越高越好。

总结一下，GPT这样的模型想要train到一个比较好的程度，本身就需要海量的数据和机器。再加上设计得比较精妙的训练过程和产生高质量训练数据的方式，两者一结合就达到了非常惊人的效果。他们十个月之前做的InstructGPT虽然训练过程也大差不大但是缺了个chatbot所以导致labeler标数据标得太枯燥了，质量不佳，产量也不佳，所以模型效果也就一般般了（无端猜测）。有钱真好啊……

================
Ref:
https://openai.com/blog/chatgpt/

On the language itself, I'm curious to see how transnational memory plays out in practice for large programs where you care about tail latency.

Clojure probably has the most used mainstream-ish transactional memory implementation and I'm told by Clojure programmers that it's rarely used.

Both the 2005 and 2022 slides make it seem like STM is core to the "language UX" in Verse in a way that it isn't Clojure, so we may actually see widespread use of transactional memory?

When looking at https://www.st.cs.uni-saarland.de/edu/seminare/2005/advanced-fp/docs/sweeny.pdf and https://simon.peytonjones.org/assets/pdfs/haskell-exchange-22.pdf, the thing that most amazes me is that, in an era when industrial research labs (MSR, Bell Labs, IBM Research, etc.) have scaled down and/or shifted much of their emphasis to shorter term projects, the company that's been willing to fund a two decade long research project for a radical new language intended to be mainstream is a game company.

#rust One weekend playing with Datafusion (https://github.com/apache/arrow-datafusion). Slightly les performant and more memory hungry than duckdb. But it's impressive anyways. These technologies used to be: 1. closed source, 2. very very very expensive. You could build a Snowflake clone with this.

#introduction Hi everyone! We're the team behind the preprint peer review platform Review Commons (https://reviewcommons.org). Review Commons provides journal-agnostic scientific peer-review of preprints submitted by their authors. Our reviews are made public and can be easily transferred to our partner journals, accelerating dissemination of peer review research and avoiding redundant rounds of peer review

When snow fell in Westminster

#London #photography #snow

amusingly, a google search for “stolen iphone shenzen” brings up a surprising number of results.

sounds like it’s not uncommon for stolen iPhones (from anywhere! e.g. from music festivals in Las Vegas or a bookstore in Canada?) to end up in Shenzen

5. Do not make your phone (or anything really) a single point of failure while traveling. I normally use my phone as my hotel key and thank god I had brought my physical key card with me that day else I could not have gotten to my laptop as fast. For me, every minute my phone stayed “unlocked” meant another minute for the thief to do damage.
6. Don’t text while waiting at a crosswalk in London I guess. :)

but some pro tips:
1. Enable Find My on your devices because if your device gets stolen you will sure as hell want to remotely wipe it.
2. If you're traveling, ensure you have a way to access Find My from your hotel. This could be a laptop or iPad you leave at the hotel.
3. Be wary of who you list as your emergency contacts as it appears a thief can view them even if your phone is put in Lost Mode.
4. Be wary of any text messages you receive after your phone is stolen.

Third, my phone's in China...?

It's been over a month and I had long assumed my phone was scrapped. But this morning I got an email saying the Activation Lock on my stolen phone was requesting my password. Huh?

I opened up Find My to see where my phone was and... it's in Shenzhen, China? Wild. I assume this is where they must finally get to scrapping my phone (unless, god forbid, they have some way to crack the activation lock).

Second, the phishing text messages.

On the second day, every one of my emergency contacts received the following text message. I opened up the URL myself and it was a full blown Find My web UI phishing page, which I assume was trying to get my Apple ID password!

For context, if an iPhone is Find My enabled and remotely wiped, it cannot be reactivated without the original Apple ID login. So unless they have my Apple ID, my phone is a brick to them. This was a step up from what I was expecting.

What happens to your smartphone when it gets stolen?

I recently had my iPhone stolen while on a business trip in London and, through the wonders of Find My, have been able to track its journey in the past few weeks. I found it to be a fascinating insight to what appears to be a rather sophisticated phone theft operation and thought I'd share!

The situation in early evening is even more extreme. The wholesale price for the hour from 6-7 pm in Great Britain is £2,585, that's just over €3,000 per MWh (€3 per kWh).

Tomorrow will be another very expensive day for electricity in Europe. This time, Great Britain stands out with a baseload price of €784 per MWh!

This climate pattern in the tropical Pacific has a big impact on global temperature. We've had 2 years of La Niña now, which kept the global average down, but..

"Tsundoku," the Japanese Word for the New Books That Pile Up on Our Shelves, Should Enter the English Language

https://www.openculture.com/2018/07/tsundoku.html

#otters #joke