John BS @johnabs@social.cleverlibre.org

@icedquinn @freemo I would imagine they actually did use "wavelets", but not in their "direct" form. More than likely they convolved a sine wave with some limiter function in the time domain to create a wavelet, which is basically just a Gaussian in the frequency domain, e.g. a Morlet wavelet of some similar equivalent. Then, assuming a Generative algorithm, they likely just composed them together to create new sounds, or match existing ones. Maybe that's how synths were made?

@freemo Thank you so much for the information and detailed breakdown of your problem! I was initially under the impression you were looking potentially variable time-scale trading (e.g. adjusting your trading frequency based on predicted trends and volatility) rather than the more static, though significantly more safe and stable, approach you're currently taking. I think it's really interesting, and I'd love to hear more about your project as it develops more; that is, if you don't mind keeping me in the loop! :D

@freemo I'm with you (I've published some work on this as well, particularly in application to NMF), but in this case of specifically looking at (what I'm assuming to be) synchronous time-series data, wouldn't you prefer an approach that allows you to work with continuous data as new information "reaches your detector" so to speak, rather than needing to recompute and compare your FT series? I guess my question would be why do it the FT way instead of the wavelet way when the latter should be more suited to this use case in general. Additionally, wouldn't it be easier to perform auto-correlation of your various data traces using wavelets in comparison to FT, which would make suggestions simpler? The real advantage of FT I would see would be ease of implementation and computational speed, which is definitely a justification I would understand, but I'm just curious precisely what your contention is with the technique.

Also, I searched for your algorithm but couldn't find it on google scholar or your Github, would you mind linking it?

@freemo Hi there Freemo, if I may make a suggestion, I would recommend trying to incorporate wavelet analysis rather than Fourier analysis for this particular application, as the wavelet transform allows for some temporal specificity on signals that vary over time, while still allowing you to access the frequency data you need. It's incredibly useful, and you may even be able to use a matrix factorization approach with the resultant data to pull out key points of interest and recommend stocks based on the behaviors of other stocks, assuming you have the processing power and access to relevant data. Here is a link to a paper I recently read on the topic regarding this application in brain networks, as well as the Wikipedia page to wavelet transforms, and a video on the application to financial time series analysis. Feel free to ask any questions you may have and I'll try to respond soon :)

https://en.wikipedia.org/wiki/Wavelet_transform

https://invidiou.site/watch?v=Z3UH3PYO_3A

https://www.researchgate.net/profile/Tianyi_Zhou8/publication/338800482_Stability-driven_Non-negative_Matrix_Factorization-based_Approach_for_Extracting_Dynamic_Network_from_Resting-State_EEG/links/5ee79db6458515814a5ebfae/Stability-driven-Non-negative-Matrix-Factorization-based-Approach-for-Extracting-Dynamic-Network-from-Resting-State-EEG.pdf

@zleap Hi Paul, I hope you don't mind if I hijack this post a bit to plug a similar tool (though it's not collaborative in the same way as Overleaf) called RMarkdown! It has nearly all the features of LaTeX (since it compiles to LaTeX as an intermediate file type) but works with R to allow for seamless integration of code, typesetting, and reproducible data analysis, via loading R-Scripts. It also gets rid of a lot of the annoying markup syntax (\begin{} etc.) in most typical use cases(though they are still usable if necessary).

For anybody who's a fan of LaTeX, I HEAVILY suggest looking at this tool, and similar ones for languages like Julia's Weave etc. In RMarkdown specifically, you can also mix multiple languages into a single RMarkdown file that runs analysis and compiles your document!

Here's a link to the documentation and overview of the tool, and I've also included a Github link to a thesis template I wrote a while back to make it much easier to automate the figure making and typesetting process, in case anybody is interested!

https://rmarkdown.rstudio.com/

https://rstudio.github.io/reticulate/articles/r_markdown.html

https://bookdown.org/yihui/rmarkdown/

https://github.com/johnabs/Thesis_Template

@hansw (Sorry this is a bit long!) I'll take a stab at this one. I used to be a TA for various courses, and I believe this lack of adventurous behavior stems from a combination of inexperience, a lack of knowledge, and a dash of learned helplessness.

1. Many first time programmers are unaware of what disastrous effect poorly implemented code may have on their machine, due to a lack of understanding what the compiler does and how most of them try to sandbox your machine from exploding if you inappropriately modify some memory. In nearly all cases, nothing bad will happen, but they are fearful of potentially negative outcomes. Experience helps assuage this fear.

2. I think they believe that learning via experience is less efficient and would rather ask and receive an answer than attempt to experiment themselves. This effect is also compounded by poor or unclear documentation. Thus, the "stack overflow will solve all my problems" mentality.

3. They haven't been in a situation with an esoteric use-case where they had to literally hack together a crap solution themselves, since they couldn't get a response.

I think once most programmers cross these hurdles, particularly the last one, they become much more comfortable with "throwing things at the wall to see what sticks". I tried to encourage my students to fail as fast as possible to reach this point of adventurous experimentation compared to the more tentative, helpless behavior.