You should make fancy visualizations of Piketty's data for the reading group to ogle.
Physicists, always trying to get something for nothing.
How far can you get doing online tutorials like Codecademy?
There may be some Coursera lectures available to accompany that Java book. I dropped out of the first version they offered.
fake accent: As someone who just did an MLS with a concentration in archives and academic libraries here's where I probably should ask you if you'd be willing to contact me off blog for some career advice. I'm having a hell of a time but would rather not hash things out on the blog.
If you do then just replace the "underscore" in my email with the thing itself.
Kid C has been doing a bit of Java (I think) at school and now they've started on Python. Think they're mostly using Khan Academy, but a friend was saying she thought code academy was pretty good. The boy keeps talking to me about this stuff. I told him I know nothing and that he should try my dad (thought they would do some programming at Easter when they had a few days together, but they just did maths instead), but still he talks. Maybe I should send him here.
I signed up for a free Project Code thing they were doing at NYPL but there was so much response that I didn't get selected. There was an exam that consisted of the first two sublessons of the first lesson for the HTML + CSS tutorials using Code Academy's platform. When I got the email saying I hadn't made the random cut I just thought fuck it. I'll do it myself. I'd been planning on using Code Academy to teach myself Python for a while and never made the leap but I've been moving along with the HTML and CSS nicely. Of course, as an avid blog commenter I've long known how to use tags to italicize or bold text and embed goatse pix helpful links and I'm familiar with some other XML based languages.
BTW, I was trying to find in the archives when Trapnel began to learn to code, it was Python, wasn't it? What did he use to learn?
D3 is the shiz-nizzle. I've been doing lots of fun stuff with it lately. Math hasn't really been an issue, d3 takes care of most of that. Here is a d3-based map I'm working on of undersea cables in the Caribbean. The next step will be to animate it so you can see the new cables being added over time.
Also, I think Java is a solid choice for you, because, based on your iOS experience, it would be helpful for you also to be able to develop in Android.
The thing I always recommend for new Python students is Learn Python the Hard Way.
11: And now there's Learn C the Hard Way. Which is really three words too long.
4: MITx's Fundamentals in Computer Science track also includes a couple of beginning software engineering courses taught in Java:
https://www.edx.org/school/mitx
I suspect they focus as much on good development life-cycle practices and processes as they do on Java (the first is not available until Q4 of this year).
Relatedly, I am currently doing the first half of BerkeleyX's Engineering Software as a Service sequence.
It's exactly the kind of intermediate online course that there are nowhere near enough of:
CS169.1x teaches the fundamentals for engineering long-lasting software using highly-productive Agile techniques to develop Software as a Service (SaaS) using Ruby on Rails. Students will understand the new challenges and opportunities of SaaS versus shrink-wrapped software. They will understand and apply fundamental programming techniques to the design, development, testing, and public cloud deployment of a simple SaaS application. Students will use best-of-breed tools that support modern development techniques including behavior-driven design, user stories, test-driven development, velocity, and pair programming. Students will learn how modern programming language features like metaprogramming and reflection can improve productivity and code maintainability.
The MIT link in 13 should be to this:
If you'd had all these skills two years ago you could have been an engineer on the project we're just wrapping up. Look, interactive d3 data mining!
Neat!
The second link in 13 also looks good. I've heard many of those words!
14: C is a very simple language. It's the pointers that kill you.
21: Mostly I couldn't get over being annoyed with the idea that I had to make a separate header file. It just goes against the grain of my inherent laziness.
The distinction supposedly being drawn in 21 is unclear to me.
23: It's very simple. This is Java 1.5's grammar dependency graph. This is C's. See?
I assume the first link in 24 is to an egg and the second shows an egg being fried.
BTW, I was trying to find in the archives when Trapnel began to learn to code, it was Python, wasn't it? What did he use to learn?
I took the first class in the MIT EdX sequence linked at 15 (although maybe it was longer then? I didn't think it was only 9 weeks). When that finished (beginning of '13), I started a bunch of Udacity ones, and also the Berkeley EdX SaaS class linked at 13, but couldn't really stick to it, and once I got a bit behind it felt hopeless.
Ooops--26 was me.
Sigh. I wish I could force myself to do what Ogged's doing, because I don't feel like my programming skills are getting much better at my new job, but it's hard to find the energy.
23: Well, you don't *have* to create a separate header file, in theory. You could just put everything in one big file, being careful to track the dependencies so that everything is defined before use. For anything bigger than a toy program or a short student project, that would produce a maintenance nightmare, but it's *possible*.
It's also possible to have a multi-file project where every outside function or variable is declared locally in every file that uses it, instead of having header files. That too is a maintenance nightmare once a project gets to a non-trivial size.
The point of the separate header file is that it describes the interface to the global functions and variables that are defined in a related file or module, so that outside code that accesses those functions or variables can determine how to interact with them correctly, without having to know the details of the implementation.
23: It's very simple. This is Java 1.5's grammar dependency graph. This is C's. See?
No. Pointers are part of C.
I am supposed to pair program / collaborate for our first homework assignment the Saas class mentioned in 13. Why use classmates when you have Unfogged? Here's one problem:
Define a method max_2_sum which takes an array of integers as an argument and returns the sum of its two largest elements. For an empty array it should return zero. For an array with just one element, it should return that element.
Answers have to be in Ruby. In the real world, I would just assign the relevant values to two variables and return sum. The instructor is really in to clarity via conciseness, so I am trying to find a way of doing the logic in one statement. Though my current solution does that, it is not very readable/transparent:
def max_2_sum(arr_num)
sum = 0
2.times { sum += arr_num.delete_at arr_num.index(arr_num.max) || next }
sum
end
Anyone know how to improve this to make it more transparent though still concise? The same code with sum += arr_num.delete arr_num.max won't work when the max value is present multiple times in the array.
Some version of chaining sort and slice is probably the way to go.
2.times { sum += arr_num.sort.pop || next}
I will put something else up during the day.
Except 'next' s/b 'break'. Done with clogging the sidebar.
I do programming* for a living, but I am depressingly un-skilled at it. If I had a lot more free time, I'd quite like to pursue the sort of thing Ogged is doing.
* officially I run projects, and do more policy and 'architect' stuff, and I have devs working for me, but in practice, I still knock up simple python and shell scripts pretty regularly, and I sometimes have to fix legacy code.
Doug Hellmann's The Python Standard Library by Example is very good, although it's a reference work rather than a course. It began as a project to blog about one python module a week, so if you're cheap you can read the whole thing on Hellmann's website. (Also, he responds to comments from newbies at the "avuncular greybeard" end of the spectrum that reaches over to R/andy B/ush at the "berserkly aggressive curmudgeon" end.)
Also, I'd like to recommend SICP, which is FREE! and also the High Scalability blog.
Hey, could someone who knows how to find them in the hoohole link the various threads there have been on this? I think this is the third, right?
For that matter, I would be grateful for a compilation of book rec threads, and that I could contribute to, because at one point I bookmarked a bunch of those. Although that's likely to be a big pile of stuff I'll be sad I'll never get to, whereas some of the programming stuff I might use.
Maybe there could be a post called "compilation of learning to code threads" and another called "compilation of book recommendation threads", so they would be easy to google. Or maybe they could be the same post. I suppose I am a FPP, and could theoretically make such a post.
2.times { sum += arr_num.sort.pop || next}
This would work for a quick hack, but you would never want to run this code in production. What if this code was in a function that was called inside a loop, and was being called against an array with 65,000 unsorted integers? Sorting can be an expensive operation, and you are sorting this array twice. What if the requirements changed so that, instead of of summing the top 2 values, you had to sum the top 2000? If your instructor is any good, that's the next part of the assignment.
Can you think of a way where you would only have to process the array one time, regardless of how many of the top values you need to sum? Or (bonus points) if you can do it without the expense of sorting the array?
I stalled in my attempt to learn Python because at no point did the tutorial give any examples of why learning Python would be useful. Strange, since apparently it is the one and only useful skill in the modern workplace, but after a couple dozen hours every example was still "how to do something that you could also do with Microsoft Excel". I think I need to have some sort of actual project in mind that the skill will allow me to do. Or take a course.
If you have a job where you have to organize lots and lots of files, depending on what "lots and lots" means, it is really nice or essential to be able to move them around in an automated way. Excel can't do that. That's most of what I use programming for. Also, sometimes I want to change the format of data that a software program spits out in a way that would be very laborious with Excel, although maybe all your systems are already optimized for the analyses you want to do. Finally, I don't know what kind of data you use, but if you are using software that has a CLI and a GUI, and you currently use the GUI to run some process on every instance of data, it would probably be a lot nicer to just control it with a loop.
(I was talking to a friend last night, describing a conversation with my boss about the time I spent a week writing a web scraper, and my friend said, "In the future, everyone will be a bad programmer." "I am the future!" I said.)
40.1: I do and that would be really useful. How do I do that?
26,27 I was under the impression that your current job is programming related and that you got it after learning Python.
You can write in C while barely understanding pointers. (I should know, as my father did it for many years.) But to grok C you need to grok pointers. They're not too hard conceptually, although I always felt that the syntax was a bit iffy (e.g. with type names, or even just remembering order of precedence for a bunch of prefix and postfix unary ops). For learning them, it might be good to start with the general concept of reference in a higher language and then work back down to pointers as a special-case lower-level version of it (and also how references of any sort in higher-level languages are implemented).
re: 39
[On preview, pwned by Tia]
Well, python is just a language. So, it might be you don't have any tasks you need to do that really require you to code, at all.
Here's a typical real-world thing that I might do:
[Apologies for length, but I thought I might as well use a real example]
Now, it would be possible to do all of the via a combination of Excel, some command line manual stuff, a bit of SQL, and some manual text editing. Perfectly do-able, if a bit time consuming, for one thing. I have done exactly that, and made heavy use of ugly hacks in Excel, when it's a one-off thing, and writing code is going to take longer than just doing it the slow way. But ... If you want to do it for dozens, or hundreds, or thousands, it's totally not.
Enter python.
So, I have some python libraries that'll handle all of the image manipulation and conversion, some simple bits of code that'll query databases and run simple transformations on it, python scripts that wrap up various command line tools so that they can be sequenced together into a workflow, and so on.
38. I have no idea how Ruby works, but in pseudocode, try:
Declare Int max1=0, max2=0, ptr=0
While ptr
If array(ptr) >= max1 Then
max2 := max1; max1:= array(ptr)
Else If array(ptr) > max2 Then
max2 := array(ptr)
End If
ptr := ptr+1
Loop
max1 := max1 + max2
End
In some languages this can be done on one line, not in others.
That's fucked up because of less than signs.
Declare Int max1=0, max2=0, ptr=0
While ptr LT size(array)
If array(ptr) GE max1 Then
max2 := max1; max1:= array(ptr)
Else If array(ptr) GT max2 Then
max2 := array(ptr)
End If
ptr := ptr+1
Loop
max1 := max1 + max2
End
The prime use case for Python getting data from one file format into another.
41: I don't know what kind of answer you want, but you could get to the point where you could do that pretty fast by taking one intro to programming class in Python (or Ruby or whatever) and learning what the os module does (or whatever it's called in other languages). In the very beginning, some tasks might feel slower than doing them by hand, but by practicing you will get to the point where you can do them faster, and automating is less error-prone. I think CN has a point -- it would be nice if introductory courses would focus more on file management because it's really the most obviously useful thing for people in a lot of professions, and it's not like it's hard or complicated or a bad way to build up fundamental concepts.
A very simple example (although a lot of this stuff is not more complicated than this, really). Say you have a bunch of files in a directory that need to have a different extension.
import os
dir = '/path/to/dir'
for f in os.listdir(dir):
if f[-5:] = 'wrong':
old = os.path.join(dir, f)
new = os.path.join(dir, f[:-5] + 'right')
os.rename(old, new)
Typing that out and running it is a lot easier than going through and renaming files, particularly if there are thousands of them.
48: I have thousands of files I have to copy and my current method involves saying "That's not in my job description" and having a guy in California do it. But eventually, he's going to get angry and they'll change my job description.
So, is that code there in Python?
39
All even half decent languages can do what Excel does. What makes "real" languages useful goes beyond that. If you get to the second half of "Learn Python the Hard Way," for example, you discover the existence of "modules" and "classes."
(I know Excel has those to some extent, or rather similar things, but from what you posted I'm guessing you don't use them much. Apologies if that assumption is wrong.)
"Modules" are a way of saying "never write a piece of code twice." Classes are a way of saying "think about the things you are working with and write code to do the things you need to do to them only once." In other words they are organizational tools and building blocks for making new code.
Python is often used for the sort of tasks that Tia mentions in 40, which most people would call "scripting."
If you want combine "scripting" stuff (fiddling with files, printing results) with the ability to plug in different "analysis" tasks, one way to do it is to write the different tasks as modules and then have a script that runs them. It means minimal changes to the script when you create a new analysis task.
The same is true of classes. A class says "A Thing1 is something you can do X and Y and Z to." Then later you create another thing that's similar, and you can say "This new Thing2 is like a
Thing1, except with it you can also do A, B, and C to it!" If (and this is often a big if) you design your universe of things carefully, you save a lot of time and effort in writing and debugging. (Often a module will consist of a bunch of classes, so you write them once and import them when you need them.)
Python, once you learn it fully, helps you be much more lazy than Excel does. Programming is all about being as lazy as you can get away with.
But not too lazy.
10 PRINT "That's not in my job description"
20 GOTO 10
Mind you, that's Basic, not python..
I had a class in BASIC. Somebody get me a ][c.
49: shell scripting is probably the right way to do that, really.
Getting to the point where you're organizing your thoughts enough to have classes and libraries/modules in a step above anything I was mentioning, and I wish I thought that way, but I don't yet (and I think part of the reason I don't is that I don't really need to). But even if I never do, the stuff I do use programming for is very labor saving. You don't have to be at the point where you're good at universe-designing to find it useful. It's definitely worth it to be a bad programmer.
46 works, but is going to run into scaling problems when the requirement changes from taking the 2 highest values to taking the top x highest values.
Part of the art of software development is anticipating the possibility of such a requirements change, judging its likelihood, and acting accordingly. As you get deeper into the field, you find that static, unchanging requirements are less and less common. I generally have the luxury of making up my own requirements, and they still need to change constantly.
54: I personally hate shell scrypting syntax and try to avoid it, although it's slowly creeping into my life. What I like about Python is that it's one tool that allows me to do the vast majority of things I want to do. Tal Yarkoni argues for the benefits of having one tool here. Python is a lot nicer and friendlier than bash syntax and reads more like pseudocode, IMO, and has more generalizable utility. When you've learned Python, you can move your files around, you can take a week to write a web scraper, and you can muck around with your file formats. (And for me, it's having another tool that I'm more comfortable with that makes shell scripting tasks more approachable.)
Also, there are now a ton of resources that make learning Python really approachable, and there is no such resource that I know of for shell scripting.
54: I don't know what that is. I'm told it can be done in SAS with macros. SAS is probably more cumbersome than other methods except for people who know SAS and not the other methods.
Getting to the point where you're organizing your thoughts enough to have classes and libraries/modules in a step above anything I was mentioning, and I wish I thought that way, but I don't yet (and I think part of the reason I don't is that I don't really need to).
That's the top down approach for library building. You might benefit from the bottom-up approach. As in, thinking "damn, this script is getting too long to maintain, maybe I can rip out these utility functions I wrote, stick them in a module, and not have to think about them right now." Or thinking, "Hey, I already wrote something that does this in that other code from last month. Maybe, instead of cutting and pasting, I could just import that script as a module."
I'm told SAS can do anything. That doesn't mean it should.
56. 46 was the result of three minutes thought (thought is irksome and three minutes is a long time). Given half an hour you would obviously make it more elegant and more scalable, easy enough if you store your high values in an array instead of independent variables. But I'm trying to cook a meal.
Tal Yarkoni is only sort of right. Having one tool for most things is nifty, but having the most appropriate tool for what you're doing is also important.
61: I'm sort of a hedgehog on some things.
Given half an hour you would obviously make it more elegant and more scalable, easy enough if you store your high values in an array instead of independent variables. But I'm trying to cook a meal.
Ah yes, the old "how much time do I want to piss away on this stupid problem" conundrum.
59: The shell is the program that runs in the terminal of your computer (the black box with the C prompt, if you're on Windows). There are different kinds of shells, which have highly related languages associated with them. On Ubuntu, which I use, the default shell is bash. So you open up a text file, start it with #! /usr/bin/bash and name it something.sh to tell the computer it's a bash script. Then you write a program in it in the language of the shell, which, IMO, is uglier, and (not IMO -- this is objective fact) less generally useful than Python. But for someone who really knows what they're doing, shell scripting is the fastest thing for a lot of tasks. The Python program I just typed out could have also been one line typed directly at the command line of your shell.
(the black box with the C prompt, if you're on Windows)
Thanks. I always called that the DOS box. I don't use it often, but I do sometimes.
Anyway, Tweety writes from the perspective of someone who's been working in software his whole life. I write from your and Cryptic Ned's perspective -- I work in the sciences and just want to program to make my job easier and don't want to be overwhelmed by how hard and weird everything is. Python is the best place to start, and the best general purpose tool. Doing some basic scripting in Python may well be an entree into lots of other competencies, but start there.
After you do some basic course, you can just google the simplest, lowest level question you have, and you'll find it on Stack Overflow. Look: https://www.google.com/search?q=how+do+I+move+a+file+python&rlz=1C1LENP_enUS494US494&oq=how+do+I+move+a+file+python&aqs=chrome..69i57j0l3.4811j0j7&sourceid=chrome&es_sm=93&ie=UTF-8
import Data.List
max_2_sum = foldr (+) 0 . take 2 . sortBy (flip compare)
29: See 43. You can "know" C and not be able to program a linked list reversal. (Trust me, I interview these people all the time.) C really is a very simple language. You can learn it (not master it) in an afternoon.
Now I suppose what Spike is saying is he never managed to do anything of any complexity in C because he couldn't wrap his head around the programming model. That's fair enough. I guess we can all agree it's somewhat better than assembly language.
That's the sort of thing I was getting at with saying I'm not a good python programmer. I understand and use modules [both 3rd party, and ones I've written], and so on, but I rarely, if ever, make use of the more 'object oriented' features that python offers. I don't think I've ever really used classes in earnest.
70.last: I love assembly language. It's nice to be so close to the actual silicon. The more abstract a language gets the less I trust it. By the time I get to object oriented languages I have completely lost faith in the giant edifice of code written by other people on top of which my own little contribution precariously totters. At that point I treat it as magic and just hope that the inevitable zillions of bugs in the compiler don't affect what I am trying to do. I think I should probably just stick to using an abacus.
55
You're absolutely right. The whole idea is to minimize the amount work you have to do. If "universe-designing" takes longer, it's not cost-effective for you. However, at some point (which varies for everyone) you say "ya know, I did that 20 times before and I'm tired of typing it" (or worse, "tired of cutting and pasting it and tweaking it to do something not that different").
Then it's worth investing the time to learn the more advanced stuff.
64 to 61
It used to be said (in ancient times) that eventually every program gets the ability to send mail added to it. This is creeping (or galloping) foxism. Emacs is a fox, Unix tools were supposed to be hedgehogs, then GNU engulfed Linux, and it's foxhogs all the way down.
68
The problem with StackOverflow, good as it is, is that reading it will scare the bejesus out of anyone who isn't a neckbearded basement dwelling geek. The answerers are not always sympathetic to the askers.
There is a bagel breakfast at this workshop. Bagel choices: sweet bagels or plain. Cream cheese choices: strawberry, cinnamon, or caramel. There is not even any goddamn plain cream cheese to put on this circular shaped fake-bagel. It's an affront.
70: Thinking about this a bit more, I wonder if this is more a function of C not being used for application development as much these days. If you're writing something new in C, you probably really do care a bit more about bit twiddling. Otherwise, you'd write it in a scripting language or Java or on the web.
Re Moby's issue: I second Tia on this. That could also be done in Python, and honestly I recommend it more than shell scripting (the built-in mini programming language of your command line to automate simple tasks). If you haven't already internalized a shell scripting language like bash scripting, it's just a pain in the ass to understand. The syntax is a little different from everything else and since it does almost everything as string manipulation it's very brittle. Having data types--even dynamically typed ones--is better than shell scripting.
Using a real scripting language is much better. By scripting language I mean an interpreted language (no need to compile your code) that it's easy to run small programs in. Python's probably your best bet, but Ruby and, less so now, Perl are also popular. (You could even use Scala or Haskell or a bunch of other more esoteric choices.) They generally have libraries to make file manipulation easy. Your code will be much easier to understand if you ever have to go back to look at it. In the unlikely event that there's something that you can only do in a shell script, most scripting languages have a function that let you run a string as a shell script. (This is like shell scripting's backticks, but so much better.)
There is a toaster, which is nice.
If the problem with shell scripting is weird syntax I am not convinced perl is the answer.
75: Maybe you could loudly make a joke about it.
Anyhow I have been dealing with giant directories and files lately (more than thousands but less than millions of files/lines in files) and have found shell scripting to be a lot more efficient than my admittedly likely-shoddy python for first-pass stuff.
Yeah, fair enough, if not for Unfogged's tag mangling I would have put blink tags around it saying DO NOT ACTUALLY PICK THIS ONE. I wanted a third example of a scripting language and Perl's always been on my list, but I should probably relegate it to a mental dustbin. Especially as a lot of its weirder bits are taken from shell scripting! More readable than bf, though.
75: That sounds like what happens whenever we get Breuggers.
80: You mean efficient in your time, not data throughput, right? I think that goes down to, again, you being an expert badass. Or badass expert.
I should say--if what you're doing can be done in one line at the command line, and doesn't need to be repeated, do that. You don't always have to reach for ("real") code. I think it's just when you start saving shell scripts in files that it gets onerous (but then again, I love aliases. I'm just a big bundle of self-contradictions today).
82.1: no, in data throughput. Also my time, probably, which goes down to any theories that I'm an expert badass being maybe not so supportable.
re: 83
I have similar issues. Often, if you are using pre-tiled images, there can be hundreds of thousands of tiny files, per book/manuscript. If you need to do anything with them, it gets pretty onerous, pretty fast. I've never really worked out a good way. I tend to use shell scripts, as the sort of things I'm doing [sorting, or renaming, or moving into a different directory hierarchy] are pretty easily shell-scriptable. It always seems slow, though.
80: Yeah, if I had to do the same thing I guarantee you my Python solution would be more efficient than my shell solution. But whatever, the best tool for the job is usually the one you're most comfortable with.
29: See 43. You can "know" C and not be able to program a linked list reversal.
Ehhh, I think the quotation marks are doing a lot of work there. Saying you can know C and not be able to reverse a linked list (I at first read this as "traverse a linked list" which would be even worse) doesn't seem like, I don't know, saying you can know C++ and not be able to compute the nth fibonacci number at compile time. (Though maybe that's old hat to C++ people now, I don't know—I don't know C++!)
Can you think of a way where you would only have to process the array one time, regardless of how many of the top values you need to sum? Or (bonus points) if you can do it without the expense of sorting the array?
Thanks! Trying to do it in one line was sort of needless wankery. First stab:
def sum_maxes (arr_nums, num_maxes)
total = 0
maxes = []
num_maxes.times { maxes.push 0}
arr_nums.each do |num|
if num > maxes[-1]
maxes.push num
maxes = maxes.shift
end
end
maxes.each { |a_max| total += a_max }
total
end
Not sure how Ruby implements array so shifting could be expensive with a large number of maxes. There's a slightly more interesting problem in the assignment that I will post here in a bit.
Oh. I wasn't even considering that--it sounds like this is a problem that needs to be fixed by having a solution, period, and we can worry about efficiency later if it becomes a problem. And anyway, these are *scientists* we're talking about. I have yet to see a counterexample to the general proposition that introducing scientists to shell scripting generates the worst code ever. (Yes, I'm a bigot.)
Unix tools were supposed to be hedgehogs, then GNU engulfed Linux, and it's foxhogs all the way down.
When did cat(1) grow the -v option?
I should say that I eventually moved to doing everything in python, because of some other crap I needed to do on every iteration. Also, I should say that everybody else in my lab does things like processing csv files with matlab so it's a little bit of a one eyed king situation.
You know what's good for processing csv files? SAS.
89: Probably rhetorical, but I was curious: over thirty years ago
68: YMMV, but I've asked a question on Stack Overflow and gotten extremely helpful answers that just kindly passed over a lot of glaring stupidity I recognized in restrospect (trying to adapt a function I found on the internet I obviously didn't really understand, using python built-ins I also didn't understand such that I was trying to do the same thing three times over, and failing each way). No one has ever been mean to me.
93: That's my experience, too. StackOverflow has been very good at using mechanistic solutions to get civility out of a population that tends not to be. It isn't perfect, but it seems like they're always experimenting and improving.
Now I suppose what Spike is saying is he never managed to do anything of any complexity in C because he couldn't wrap his head around the programming model.
Actually, my problem with C is that getting work done bit-twiddling takes a lot of patience, and I am not a patient man. "Do I really have to maintain a separate header file? And manage my own damn garbage collection too?"
I crave higher levels of abstraction.
Shell is perhaps the highest available level of abstraction, but it is something of a dark art. And its not very portable and doesn't scale well into large programs.
For really simple batch file stuff, I've taken the lazy path of setting up a spreadsheet to generate single line commands where the only difference between lines may be a file name and a setting or two that has to be determined case by case and then pasted that column into a shell script.
96 is what I want to do. I still do a great deal of SAS coding that way instead of setting up an array or a macro. This annoys the junior people when they have to go back and adapt my code.
Or let SAS call a dataset with the file names and spit out shell script.
A Unix-like system is basically a fox suit full of hedgehogs. (In fact, I think you can argue that the whole of CS is an effort to make a fox out of hedgehogs.)
95: I think being forced to separate the signature/interface/data definition and implementation of your code is generally a good thing, but it is annoying in the moment.
I don't think shell is very high abstraction; you're almost always just working with strings concatenated together. If you forgot to escape something (or forgot how many nested levels of escapes you should have), things will break in silly, unpredictable ways. If you want more abstraction, I think you can run Haskell and some of the MLs interpretively, or even us a (domain specific subset of) Scala. That might be the point where the time efficiency of your program starts to get worrisome, though.
26,27 I was under the impression that your current job is programming related and that you got it after learning Python.
It is; I do write code, but generally simple workarounds to get our product working in the desired way for clients. And I got it after doing a web developer bootcamp thing this fall. But the nature of the job means that becoming significantly better will require real initiative on my part, and I lack all initiative.
I don't know Ruby, but 87 looks pretty close to me. I'm not sure, but I think you might have a bug that I can't quite place my finger on. Try testing it with arr_nums = [3,7,4,6,5] and num_max = 3.
It looks to me like it needs a bunch of semicolons.
I don't think shell is very high abstraction; you're almost always just working with strings concatenated together.
The strings are just the glue. The abstraction is that what you are gluing together are potentially massive programs, each one designed in its own way to do its own thing.
Re: 30, you can do it in one line using inject. Writing it is left as an exercise for the reader (aka I should be mowing the lawn right now.)
Huh, I guess we just have different definitions: when I hear "high abstraction" I think "approaching category theory." Any language is capable of calling massive programs, but how you do it matters and shell scripting involves a communication channel that I find very low level and only barely structurally related to the data you're trying to pass about. It's not a very nice API.
It looks to me like it needs a bunch of semicolons.
Wait until a Rubyist tells you about "poetry mode"
shell scripting involves a communication channel that I find very low level and only barely structurally related to the data you're trying to pass about
I think PowerShell solves this issue in a very interesting way - you get to pass full-blown objects around instead of strings. But I've never really used it as .NET is somewhat alien to me.
62 But I'm trying to cook a meal.
I bet you've cooked the same meal many times before. That's exactly the sort of problem that programming is supposed to solve, isn't it? You should have a meal-cooking module by now.
processing csv files with matlab
It's not like I'm surprised, but this is insane.
You should have a meal-cooking module by now.
Its called a microwave.
I've noticed that all the younger people I work with say "C" when they mean "C++". Is that a thing everyone does now? It keeps confusing me.
112: Gotta save those characters, bro, they're precious. When talking about I was referring to vanilla C, but you're right that the distinction is often elided.
Some people say Java when they mean JavaScript. Some people just don't understand.
Some people say java when the mean coffee, but I feel like if you did that now you'd sound all late '90s/early 2000s.
93: Good that your experience with it has been positive! I land on there from a lot of Google searches for obscure stuff and see too much of the other.
99: In fact, I think you can argue that the whole of CS is an effort to make a fox out of hedgehogs.
"The OS knows many things, but the bit knows one small thing."
Followup to 87:
That code was in fact badly broken (was only looking at the last element of the maxes, for ex, instead of all of the maxes). Also initializing maxes to 0s breaks things when the array has negative numbers. I have a cumbersome solution that works, but am going to try the suggestion in 105 now to use 'inject'.
The more interesting problem I mentioned was this:
Define a method sum_to_n? which takes an array of integers and an additional integer, n, as arguments and returns true if any two distinct elements in the array of integers sum to n. An empty array or single element array should both return false.
I solved it using:
http://www.ruby-doc.org/core-2.1.1/Array.html#method-i-permutation
def sum_to_n?(arr_num, n)
arr_num.permutation(2).to_a.each { |perm_pair| return true if perm_pair[0] + perm_pair[1] == n}
false
end
There is probably a way of doing it that doesn't have the overhead of creating the permutations.
essear, do you know what's going on with the new BICEP2 rumors?
There is probably a way of doing it that doesn't have the overhead of creating the permutations.
If omit '#to_a' and call '#each' directly on the result of the permutation method you'll only create as many pairs as are needed to check the result.
If omit '#to_a' and call '#each' directly on the result of the permutation method you'll only create as many pairs as are needed to check the result.
Ooh, that's cool.
120: Not something 69s are usually accused of.
120: It translates pretty directly to most languages that provide function idioms. E.g., in Ruby:
def foo(ary)
ary.sort{|x,y| y x}.take(2).reduce(0, &:+)
end
Er, make that:
def foo(ary)
ary.sort{|x,y| y <=> x}.take(2).reduce(0, &:+)
end
This thread inspired me! I made a file called uf, for utility functions, and I put a function in there I know I have lots of occasions to call, a function I found on the internet that takes a path and returns all the parts of it as a list. I will put other useful stuff in there and start importing it as a module in the future.
Further to 121 I have to spend a lot more time really understanding Enumerable.
120: This is better if you're trying to get the top five items.
Nosflow's 69 is elegant, but I think this is a more interesting problem if you do Spike's extension of it in 38 where you can't sort. You can take it from an O(nlog(n)) problem to O(n*m), where m is the number of top elements that you want to get the sum of.
a function I found on the internet that takes a path and returns all the parts of it as a list
Is this something other than os.path.split()?
Is this something other than os.path.split()?
robert_frost.roads.diverge also works.
Oh derp, I forgot that os.path.split() just gives you a pair back. $PATH.split(os.path.sep) is what I was thinking of.
118: More than most of the people talking about it on the internet, yes.
Further to 131, if you want to know more, feel free to email me. I've been told that some details from the one critic I really trust will go public tomorrow, so things should be clearer then.
Ooh, can we at least hear the rumors Upetgi is privy to?
127: If you're really concerned with running time it should be possible to do something with insert sort that is O(n*log(m)).
133: Check your favorite particle physics or cosmology blog. If you don't have a favorite, the correct choice of favorite is called "resonances" but with the "a" doubled. I would prefer that no one link it here because its author would probably guess my identity within about a minute of reading my comments.
Or not. Insertion sort isn't what I remembered. You need to maintain a sorted list of the m highest numbers encountered, occasionally adding a new element and dropping the lowest.
Yeah, I was thinking insertion sort did something heapish.
Maintaining the list of highest numbers as a binary tree would get you the log(m) bit.
Contra 127 and my 134, I think O(n*m*log(m)) is the best you can do.
Wait, that's not right at all. I really have to stop thinking aloud.
Although, in most cases, you are better off just doing a sort, and then adding up the top m values. If you are only running the sort once, the performance hit is likely to be acceptable. Getting it down to O(n*log(m)) is nice, but adds complexity to the code and takes developer time, which is usually quite a bit more valuable than additional CPU cycles. This is the kind of optimization that you would come back to deal with toward the end of the project, if you aren't happy with the overall performance of the application, and if you have plausible evidence that points to this particular code as a source of delay.
142: Agreed, I would never do that in practice, but for toy CS problems I thought it was canonical to do stupid overly early optimization 'cause fun.
I'll just wait until tomorrow, as I'll be flying the whole time between now and then anyway. I was really hoping it'd pan out, not just for science reasons but also because I have a friend on the collaboration who deserves some good luck to counteract the bad (most recently the government shutdown delaying his main experiment by a year).
Semi-thread jack: Say a fella was thinking about saying "fuck it" concerning his first career. (OK, it's me.) I've been dicking around with website layout, presentation design, etc., for a few years and find I've been happier doing that than any other part of my job. I'm most interested in the interface between graphic design/UX/informational organization. Basically, I think I (may, possibly) want to refocus my career on making interactive thingies pretty and intuitive.
How do I go about learning to do that? What programs do I need to learn? Will I have to learn how to code? (Objective C for app development?)
Your thoughts are appreciated.
You might be interested at poking at this program at Stanford, although I assume "drop everything and move to Palo Alto to get a master's degree" isn't on the bucket list.
There is a category of "web designers" who do most of their work in Photoshop, before handing it over to "web developers" for implementation. You might go that route, although the pay is on the bottom end of the IT scale.
I would focus on the HTML5 stack, rather than Objective C, because Objective C pretty much locks you into iOS app development, which is something of a ghetto.
If you want to learn, the best thing to do is envision a project that encompasses the things you want to learn, and then do it. Chances are you will do a shitty job of it, but you will pick up a lot of skills. If you keep repeating that process, eventually your projects will turn out less shitty, and you will have a portfolio you can show to potential employers.
145: My girlfriend is interested in exactly the sort of thing you're talking about, and just completed this course. If you're interested, I can hook you two up and you can talk to her about it.
Oh yeah, and you don't really need to know how to code to do that kind of work. You'd need to pick up Photoshop and HTML/CSS, but that's really it.
Thanks guys. Going back to school or doing an intensive bootcamp is probably off the table barring involuntary unemployment, both from an expense standpoint and time commitment perspective.
Earning potential is important--I earn like 5 times what my wife does, so we can't really fall back on her income (my fevered middle of the night panic attacks, let me show you them). I'm willing to take a step or maybe even two backwards, but I anticipate remaining the primary breadwinner.
Will noodle. Josh, I may want to talk to your GF.
Is there any need for UX design within your current field that you could transition to? That would give you a leg up in finding a more lucrative niche than graduating from boot camp and signing on with a commodity web design shop.
I doubt it. I do general Mar Comm for Medical Device companies.
Might there be a survey design angle in that?
112: When was the last time you used a C compiler that couldn't also do C++?
154: Even if you use the same tools, the acts of programming in them are very different.
Learning C++ as a first language for a junior-high kid: insane. Right? (Asking for a friend's kid.)
153: I'm not sure? By Mar Comm I mean Marketing Communications--I work with graphic designers, web folks, app developers, animators, etc. to develop brochures, websites, app, animations, etc. that are used as marketing tools by our organization. We occasionally do a survey, but they generally don't have to be pretty and are pretty off the shelf SurveyMonkey deals.
Unless they're extremely self-motivated and want to learn it to do something specific (say, game programming), I don't think that's the wisest choice.
Yeah, pretty insane. A decent C++ programmer is one who understands all the things that are troublesome about the language and can cope with them (This is probably true for most programming languages, but C++ has an unusually large number of ways to do things badly). This means that it should ideally not be your first or second or third language. If you go into C++ and become wild-eyed at all the great things it lets you do, as opposed to boggling at all the different ways it lets you hurt yourself, you've done it wrong.
C++ was the first language I learned. It was fairly confusing. Wouldn't recommend it to others.
145: I have a friend who may have some valuable perspectives on that, as he does many of the same things. I'll message you elsewhere.
C++ was the first language I learned.
"My parents spoke exclusively C++ with me for about the first eighteen months."
std::cout << baby::languageFilter("Go the f--- to sleep!") << std::endl;
If you want to try out drawing wireframes, I really like Balsamiq. It's not free but you can continue drawing after the trial ends, you just can't save.
If you want to learn, the best thing to do is envision a project that encompasses the things you want to learn, and then do it. Chances are you will do a shitty job of it, but you will pick up a lot of skills. If you keep repeating that process, eventually your projects will turn out less shitty, and you will have a portfolio you can show to potential employers.
Just wanted to draw attention to this, from Spike's 147, as it's good advice in general.
168 is however not recommended for some professions, including (but not limited to): airline pilot, heart surgeon, passenger ferry captain, and armoured brigade commander. ("...But my fifth deliberate assault on a fortified position actually went comparatively well. Look!")
I work with graphic designers, web folks, app developers, animators, etc. to develop brochures, websites, app, animations, etc. that are used as marketing tools by our organization.
This sounds like a reasonable base to start from. Perhaps there are some information design aspects of this you could take over for yourself, and figure out the technology you need along the way?
I don't even really know what UX and wireframes are, but I heard a developer friend talking to his UX friend about this app called Pop that lets you draw prototypes on paper.
Thats cool. Reminds me of Balsamiq Mockups, except with paper.
I guess 172 adds value to 166 by actually providing a link.
Both Balsamic and POP look pretty awesome--thanks!
And thanks to everyone for all the other advice on the thread. This has been super helpful.
P.S. the developer friend works for OkCupid, and I have already suggested features that got implemented ("search for people who answered a question a particular way" was mine) so if you have anything you'd like to see in the Cupe, pass it my way.
176: Not that this is relevant to my interests, but one request I've seen elsewhere is "block messages from anyone who answered a question a particular way".
I have just started Powershell. Python is probably easier, but I'd have to install it.
176: Have an RPG version of the site for people who are off the dating market but think OKC is a cool site, with no way of anything bad happening.
No. I successfully copied a folder, but not the shit in it.
I have successfully copied a file.
181: This is one of those questions like "Are those real clowns, or are they just dressed up as clowns?" What's the difference between flirting with people online and playing an RPG where you pretend to?
What's the difference between flirting with people online and playing an RPG where you pretend to?
If its an RPG, you aren't cheating.
I always get so mad when people use that OKCupid mod that makes all the pretensions transparent. Way to take all the fun out of it!
So, if I just write a text file with all the copying I want, how do I tell power shell that I would like it to run it as a program?
"Are those real clowns, or are they just dressed up as clowns?"
Look, I told you guys I didn't have a lot of time to get dating right.
I can just call it something.ps1 and it will run when I type that file name and path? Let's try it and see.
I need to pretend I'm Oklahoma and call a guy who can let me change something called my execution policy.
I have successfully copied a file with a script, so far as I can tell.
But I'm disconnected from all the network drives for some reason. I can get to them, but powershell can't.
Have you tried reversing the polarity on the photon torpedoes?
I've now successfully copied a file with a script from the drive I want to copy files from to the drive I want to copy files to. Now I just need to write SAS code to write the copy statements.
I have now written and successfully executed a script that takes multiple files, renames them, and copies them to a new directory in their proper places.
I have strange new powers, is what I'm saying. I'm going to go look around and see if I don't come across a late-80s Sean Young in a skin-tight rubber suit holding a knife at my back.
197/198: This is the best thing about programming.
199: in a few minutes, Moby is going to find a bug and feel like the biggest idiot on the planet. Then fix it and feel like a genius again.
200: I went through that process twice. Now I'm trying to figure out how to get sufficient bandwidth to get several hundred gigabytes to another city. I've got such a bad connection, I'm thinking it's going to involve a hard drive and FedEx.
Hundreds of gigabytes? Hard-drive and FedEx, yes. Often the quickest.
http://techinternets.com/copy_calc?do
At 25Mb/s, 250GB or so, will take about a day. Assuming you can keep that sustained rate up. In practice, once you get over a few hundred GB, post does tend to be faster.
I'm using our Dropbox clone to send samples, then I'll do the whole thing on a drive.
then I'll do the whole thing on a drive.
Just use FedEx like ttaM said.
Scaling up is working well. I successfully copied 5,500 files comprising nearly 2GB.
late-80s Sean Young
No, you have to start with Kelly LeBrock. It's in the manual.
And now, I have written SAS code that writes script code. There's probably a better tool and somebody competent would probably have no written code that takes three lines for each file to be copied, but I can't figure out a single reason why having a 4,000 program is a problem.
This is a lot more learning that is typical for me here.
Nope. the SAS code that writes script code that doesn't work. Stupid script code.
Now it works. Apparently, script code can't guess that 'M12' was supposed to be '12M'.
This concludes me live commenting my job. For today.
When they looked into mass file transfer for a project expected to generate a terabyte every few days for a while at my old job, hard drives came out ahead. Lots of small files really slowed things down.